• Title/Summary/Keyword: storage overhead

Search Result 181, Processing Time 0.02 seconds

A Real-time Dynamic Storage Allocation Algorithm Supporting Various Allocation Policies (다양한 할당 정책을 지원하는 실시간 동적 메모리 할당 알고리즘)

  • 정성무
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.25 no.10B
    • /
    • pp.1648-1664
    • /
    • 2000
  • This paper proposes a real-time dynamic storage allocation algorithm QSHF(quick-segregated-half-fit) that provides various memory allocation policies. that manages a free block list per each word size for memory requests of small size good(segregated)-fit policy that manages a free list per proper range size for medium size requests and half-fit policy that manages a free list per proper range size for medium size requests and half-fit policy that manages a free list per each power of 2 size for large size requests. The proposed algorithm has the time complexit O(1) and makes us able to easily estimate the worst case execution time(WCET). This paper also suggests two algorithm that finds the proper free list for the requested memory size in predictable time and if the found list is empty then finds next available non-empty free list in fixed time. In order to confirm efficiency of the proposed algorithm we simulated the memory utilization of each memory allocation policy. The simulation result showed that each policy guarantees the constant WCET regardless of memory size but they have trade-off between memory utilization and list management overhead.

  • PDF

Design and Implementation of Kernel-Level Split and Merge Operations for Efficient File Transfer in Cyber-Physical System (사이버 물리 시스템에서 효율적인 파일 전송을 위한 커널 레벨 분할 및 결합 연산의 설계와 구현)

  • Park, Hyunchan;Jang, Jun-Hee;Lee, Junseok
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.14 no.5
    • /
    • pp.249-258
    • /
    • 2019
  • In the cyber-physical system, big data collected from numerous sensors and IoT devices is transferred to the Cloud for processing and analysis. When transferring data to the Cloud, merging data into one single file is more efficient than using the data in the form of split files. However, current merging and splitting operations are performed at the user-level and require many I / O requests to memory and storage devices, which is very inefficient and time-consuming. To solve this problem, this paper proposes kernel-level partitioning and combining operations. At the kernel level, splitting and merging files can be done with very little overhead by modifying the file system metadata. We have designed the proposed algorithm in detail and implemented it in the Linux Ext4 file system. In our experiments with the real Cloud storage system, our technique has achieved a transfer time of up to only 17% compared to the case of transferring split files. It also confirmed that the time required can be reduced by up to 0.5% compared to the existing user-level method.

Design and Evaluation of an Efficient Flushing Scheme for key-value Store (키-값 저장소를 위한 효율적인 로그 처리 기법 설계 및 평가)

  • Han, Hyuck
    • The Journal of the Korea Contents Association
    • /
    • v.19 no.5
    • /
    • pp.187-193
    • /
    • 2019
  • Key-value storage engines are an essential component of growing demand in many computing environments, including social networks, online e-commerce, and cloud services. Recent key-value storage engines offer many features such as transaction, versioning, and replication. In a key-value storage engine, transaction processing provides atomicity through Write-Ahead-Logging (WAL), and a synchronous commit method for transaction processing flushes log data before the transaction completes. According to our observation, flushing log data to persistent storage is a performance bottleneck for key-value storage engines due to the significant overhead of fsync() calls despite the various optimizations of existing systems. In this article, we propose a group synchronization method to improve the performance of the key-value storage engine. We also design and implement a transaction scheduling method to perform other transactions while the system processes fsync() calls. The proposed method is an efficient way to reduce the number of frequent fsync() calls in the synchronous commit while supporting the same level of transaction provided by the existing system. We implement our scheme on the WiredTiger storage engine and our experimental results show that the proposed system improves the performance of key-value workloads over existing systems.

Performance Evaluation of WAN Storage Migration Scheme for Cloud Computing Environment (클라우드 컴퓨팅 환경을 위한 WAN 스토리지 이주 기법 성능평가)

  • Chang, Jun-Hyub;Lee, Won-Joo;Jeon, Chang-Ho
    • Journal of the Korea Society of Computer and Information
    • /
    • v.17 no.5
    • /
    • pp.1-7
    • /
    • 2012
  • In this paper, we design and implement the simulator for WAN storage replication model performance evaluation in cloud computing environment. Each cloud of simulator is composed of virtual machine emulator and storage emulator. The virtual machine emulator is composed of read/write ratio module, the read/write sequence combination module, and the read/write request module. The storage emulator is composed of storage management module, data transfer module, read/write operations module, and overhead processing module. Using the simulator, we evaluate performance of migration scheme, pre-copy and the post-copy, considering about read/write ratio, network delay, and network bandwidth. Through simulation, we have confirmed that the average migration time of pre-copy was decreased proportional to the read operation. However, average migration time of post-copy was on the increase. Also, the average migration time of post-copy was increased proportional to the network delay. However, average migration time of pre-copy was shown uniformly. Therefore, we show that pre-copy model more effective to reduce the average migration time than the post-copy model. The average migration time of pre-copy and post-copy were not affected by the change of network bandwidth. Therefore, these results show that selects the storage replication model to be, the network bandwidth know not being the important element.

Enhancing Dependability of Systems by Exploiting Storage Class Memory (스토리지 클래스 메모리를 활용한 시스템의 신뢰성 향상)

  • Kim, Hyo-Jeen;Noh, Sam-H.
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.37 no.1
    • /
    • pp.19-26
    • /
    • 2010
  • In this paper, we adopt Storage Class Memory, which is next-generation non-volatile RAM technology, as part of main memory parallel to DRAM, and exploit the SCM+DRAM main memory system from the dependability perspective. Our system provides instant system on/off without bootstrapping, dynamic selection of process persistence or non-persistence, and fast recovery from power and/or software failure. The advantages of our system are that it does not cause the problems of checkpointing, i.e., heavy overhead and recovery delay. Furthermore, as the system enables full application transparency, our system is easily applicable to real-world environments. As proof of the concept, we implemented a system based on a commodity Linux kernel 2.6.21 operating system. We verify that the persistence enabled processes continue to execute instantly at system off-on without any state and/or data loss. Therefore, we conclude that our system can improve availability and reliability.

A Data Mining Approach for Selecting Bitmap Join Indices

  • Bellatreche, Ladjel;Missaoui, Rokia;Necir, Hamid;Drias, Habiba
    • Journal of Computing Science and Engineering
    • /
    • v.1 no.2
    • /
    • pp.177-194
    • /
    • 2007
  • Index selection is one of the most important decisions to take in the physical design of relational data warehouses. Indices reduce significantly the cost of processing complex OLAP queries, but require storage cost and induce maintenance overhead. Two main types of indices are available: mono-attribute indices (e.g., B-tree, bitmap, hash, etc.) and multi-attribute indices (join indices, bitmap join indices). To optimize star join queries characterized by joins between a large fact table and multiple dimension tables and selections on dimension tables, bitmap join indices are well adapted. They require less storage cost due to their binary representation. However, selecting these indices is a difficult task due to the exponential number of candidate attributes to be indexed. Most of approaches for index selection follow two main steps: (1) pruning the search space (i.e., reducing the number of candidate attributes) and (2) selecting indices using the pruned search space. In this paper, we first propose a data mining driven approach to prune the search space of bitmap join index selection problem. As opposed to an existing our technique that only uses frequency of attributes in queries as a pruning metric, our technique uses not only frequencies, but also other parameters such as the size of dimension tables involved in the indexing process, size of each dimension tuple, and page size on disk. We then define a greedy algorithm to select bitmap join indices that minimize processing cost and verify storage constraint. Finally, in order to evaluate the efficiency of our approach, we compare it with some existing techniques.

Design and Implementation of Flash Translation Layer with O(1) Crash Recovery Time (O(1) 크래시 복구 수행시간을 갖는 FTL의 설계와 구현)

  • Park, Joon Young;Park, Hyunchan;Yoo, Chuck
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.10
    • /
    • pp.639-644
    • /
    • 2015
  • The capacity of flash-based storage such as Solid State Drive(SSD) and embedded Multi Media Card(eMMC) is ever-increasing because of the needs from the end-users. However, if a flash-based storage crashes, such as during power failure, the flash translation layer(FTL) is responsible for the crash recovery based on the entire flash memory. The recovery time increases as the capacity of the flash-based storages increases. We propose O1FTL with O(1) crash recovery time that is independent of the flash capacity. O1FTL adopts the working area technique suggested for the flash file system and evaluates the design on a real hardware platform. The results show that O1FTL achieves a crash recovery time that is independent of the capacity and the overhead, in terms of I/O performance, and achieves a low P/E cycle.

A Study on KSI-based Authentication Management and Communication for Secure Smart Home Environments

  • Ra, Gyeong-Jin;Lee, Im-Yeong
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.12 no.2
    • /
    • pp.892-905
    • /
    • 2018
  • In smart home environment, certificate based signature technology is being studied by communication with Internet of Things(IoT) device. However, block - chain technology has attracted much attention because of the problems such as single - point error and management overhead of the trust server. Among them, Keyless Signature Infrastructure(KSI) provides integrity by configuring user authentication and global timestamp of distributed server into block chain by using hash-based one-time key. In this paper, we provide confidentiality by applying group key and key management based on multi - solution chain. In addition, we propose a smart home environment that can reduce the storage space by using Extended Merkle Tree and secure and efficient KSI-based authentication and communication with enhanced security strength.

A New Group Key Management Protocol for WSN

  • Gerelbayar, Tegshbayar;Lee, Sang-Min;Park, Jong-Sou
    • Convergence Security Journal
    • /
    • v.8 no.1
    • /
    • pp.143-152
    • /
    • 2008
  • Sensor networks have a wide spectrum of military and civil applications, particularly with respect to security and secure keys for encryption and authentication. This thesis presents a new centralized approach which focuses on the group key distribution with revocation capability for Wireless Sensor Networks. We propose a new personal key share distribution. When utilized, this approach proves to be secure against k-number of illegitimate colluding nodes. In contrast to related approaches, our scheme can overcome the security shortcomings while keeping the small overhead requirements per node. It will be shown that our scheme is unconditionally secure and achieves both forward secrecy and backward secrecy. The analysis is demonstrated in terms of communication and storage overheads.

  • PDF

Applying Hebbian Theory to Enhance Search Performance in Unstructured Social-Like Peer-to-Peer Networks

  • Huang, Chester S.J.;Yang, Stephen J.H.;Su, Addison Y.S.
    • ETRI Journal
    • /
    • v.34 no.4
    • /
    • pp.591-601
    • /
    • 2012
  • Unstructured peer-to-peer (p2p) networks usually employ flooding search algorithms to locate resources. However, these algorithms often require a large storage overhead or generate massive network traffic. To address this issue, previous researchers explored the possibility of building efficient p2p networks by clustering peers into communities based on their social relationships, creating social-like p2p networks. This study proposes a social relationship p2p network that uses a measure based on Hebbian theory to create a social relation weight. The contribution of the study is twofold. First, using the social relation weight, the query peer stores and searches for the appropriate response peers in social-like p2p networks. Second, this study designs a novel knowledge index mechanism that dynamically adapts social relationship p2p networks. The results show that the proposed social relationship p2p network improves search performance significantly, compared with existing approaches.