• Title/Summary/Keyword: storage overhead

Search Result 181, Processing Time 0.029 seconds

Torus Network Based Distributed Storage System for Massive Multimedia Contents (토러스 연결망 기반의 대용량 멀티미디어용 분산 스토리지 시스템)

  • Kim, Cheiyol;Kim, Dongoh;Kim, Hongyeon;Kim, Youngkyun;Seo, Daewha
    • Journal of Korea Multimedia Society
    • /
    • v.19 no.8
    • /
    • pp.1487-1497
    • /
    • 2016
  • Explosively growing service of digital multimedia data increases the need for highly scalable low-cost storage. This paper proposes the new storage architecture based on torus network which does not need network switch and erasure coding for efficient storage usage for high scalability and efficient disk utilization. The proposed model has to compensate for the disadvantage of long network latency and network processing overhead of torus network. The proposed storage model was compared to two most popular distributed file system, GlusterFS and Ceph distributed file systems through a prototype implementation. The performance of prototype system shows outstanding results than erasure coding policy of two file systems and mostly even better results than replication policy of them.

An Analysis of the Overhead of Multiple Buffer Pool Scheme on InnoDB-based Database Management Systems (InnoDB 기반 DBMS에서 다중 버퍼 풀 오버헤드 분석)

  • Song, Yongju;Lee, Minho;Eom, Young Ik
    • Journal of KIISE
    • /
    • v.43 no.11
    • /
    • pp.1216-1222
    • /
    • 2016
  • The advent of large-scale web services has resulted in gradual increase in the amount of data used in those services. These big data are managed efficiently by DBMS such as MySQL and MariaDB, which use InnoDB engine as their storage engine, since InnoDB guarantees ACID and is suitable for handling large-scale data. To improve I/O performance, InnoDB caches data and index of its database through a buffer pool. It also supports multiple buffer pools to mitigate lock contentions. However, the multiple buffer pool scheme leads to the additional data consistency overhead. In this paper, we analyze the overhead of the multiple buffer pool scheme. In our experimental results, although multiple buffer pool scheme mitigates the lock contention by up to 46.3%, throughput of DMBS is significantly degraded by up to 50.6% due to increased disk I/O and fsync calls.

An Efficient Packetization Method for the Real-time Internet Video Transmission (실시간 인터넷 동영상 전송을 위한 효율적인 패킷화 기법)

  • Kim Hyo-Hyun;Yoo Kook-Yeol
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.31 no.6C
    • /
    • pp.614-622
    • /
    • 2006
  • In this paper, we propose an efficient packetization method to reduce the packetization overhead. For the purpose, we firstly verify the relationship between packet length and packet loss rate. The empirical results show that as the packet length is larger than the path MTU, the packet loss rate is drastically increased, producing poor visual quality at the receiver side. However, as the length of the packet is reduced, we should transmit more packets per frame and the packetization overhead will be increased. This increase in the packetization overhead reduces the number of bits allocated to the video data, resulting in the low visual quality. Therefore, each packet should be packetized to have the packet length close to the path MTU. In this paper, we show that the this process of the packetization with the constraint on the packet length is very similar to the dynamic storage allocation in the operating system. We had thoroughly surveyed the dynamic storage allocation methods used in the recent operating systems and propose to use the allocation methods for the video packetization. We empirically show that the proposed method can reduce the packetization overhead upto 28.3%, compared with the conventional sequential packetization method which have been widely used in Internet video transmission.

Garbage Collection Technique for Reduction of Migration Overhead and Lifetime Prolongment of NAND Flash Memory (낸드 플래시 메모리의 이주 오버헤드 감소 및 수명연장을 위한 가비지 컬렉션 기법)

  • Hwang, Sang-Ho;Kwak, Jong Wook
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.11 no.2
    • /
    • pp.125-134
    • /
    • 2016
  • NAND flash memory has unique characteristics like as 'out-place-update' and limited lifetime compared with traditional storage systems. According to out-of-place update scheme, a number of invalid (or called dead) pages can be generated. In this case, garbage collection is needed to reclaim invalid pages. Because garbage collection results in not only erase operations but also copy operations of valid (or called live) pages to other blocks, many garbage collection techniques have proposed to reduce the overhead and to increase the lifetime of NAND Flash systems. This techniques sometimes select victim blocks including cold data for the wear leveling. However, most of them overlook the cost of selecting victim blocks including cold data. In this paper, we propose a garbage collection technique named CAPi (Cost Age with Proportion of invalid pages). Considering the additional overhead of what to select victim blocks including cold data, CAPi improves the response time in garbage collection and increase the lifetime in memory systems. Additionally, the proposed scheme also improves the efficiency of garbage collection by separating cold data from hot data in valid pages. In experimental evaluation, we showed that CAPi yields up to, at maximum, 73% improvement in lifetime compared with existing garbage collections.

Implementation of Optical-based Measuring Instrument for Contact Wire Geometry in Electric Railway (전기철도의 전차선로 형상검측을 위한 광학기반 검측 장치 구현)

  • Park, Young;Cho, Yong-Hyeon;Jung, Ho-Sung;Lee, Ki-Won;Kim, Hyung-Chul;Kwon, Sam-Young;Park, Hyun-June;Kim, Won-Ha
    • Journal of the Korean Institute of Electrical and Electronic Material Engineers
    • /
    • v.21 no.9
    • /
    • pp.868-871
    • /
    • 2008
  • We propose an optical-based measuring instrument of catenary system in electric railway. This system was made to utilize line scan camera as inspecting system to measure the stagger and height of overhead contact wire in railway and composed with optical type source and FPGA-based image acquisition system with PCI slot. Vision acquisition software has been used for the application to programming interface for image acquisition, display, and storage with a frequency of sampling. To check the validity of our approach for the intended application, we monitored height and stagger in the overhead wire of a high-speed catenary system in Korea. The proposed optical-based measuring instrument to measure the contact wire geometry such as the hight and stagger shows promising on-field applications for online condition motoring. We expect that a new generation of real-time instruments with demanding various conditions motoring requirement in railway can be easily integrated into optical-based measuring instrument system.

Implementation of Real-Time Monitoring System for Overhead Contact Wire in Electric Railway (전차선로 검측을 위한 실시간 화상처리 시스템 구현)

  • Park, Young;Cho, Young-Hyeon;Lee, Ki-Won;Kwon, Sam-Young;Park, Hyun-Jun;Jang, Dong-Uk
    • Proceedings of the Korean Institute of Electrical and Electronic Material Engineers Conference
    • /
    • 2006.06a
    • /
    • pp.543-544
    • /
    • 2006
  • This paper describes a simple real-time monitoring system for use in measurement subsystem of contact wire and geometry of overhead contact wire in electric railway. The system has been consists of a high speed CMOS camera with resolution $1024\;{\times}\;1280$ pixels, line type laser source with a power equal to 300 mW, and PC-based image acquisition system with PCI Express slot. National instrument LabVIEW (8.0) and vision acquisition software have been used in application programming interface for image acquisition, display, and storage with a frequency of sampling of 500 acquisitions per second.

  • PDF

Implementation of Storage Service Protocol on Infiniband based Network (인피니밴드 네트웍에서 RDMA 기반의 저장장치 서비스 프로토콜개발)

  • Joen Ki-Man;Park Chang-Won;Kim Young-Hwan
    • 한국정보통신설비학회:학술대회논문집
    • /
    • 2006.08a
    • /
    • pp.77-81
    • /
    • 2006
  • Because of the rapid increasing of network user, there are some problems to tolerate the network overhead. Recently, the research and technology of the user-level for high performance and low latency than TCP/IP which relied upon the kernel for processing the messages. For example, there is an Infiniband technology. The Infiniband Trade Association (IBTA) has been proposed as an industry standard for both communication between processing node and I/O devices and for inter-processor communication. It replaces the traditional bus-based interconnect with a switch-based network for connecting processing node and I/O devices. Also Infiniband uses RDMA (Remote DMA) for low latency of CPU and OS to communicate between Remote nodes. In this paper, we develop the SRP (SCSI RDMA Protocol) which is Storage Access Protocol on Infiniband network. And will compare to FC (Fibre Channle) based I-SCSI (Internet SCSI) that it is used to access storage on Etherent Fabric.

  • PDF

Design and Implementation of Inline Data Deduplication in Cluster File System (클러스터 파일 시스템에서 인라인 데이터 중복제거 설계 및 구현)

  • Kim, Youngchul;Kim, Cheiyol;Lee, Sangmin;Kim, Youngkyun
    • KIISE Transactions on Computing Practices
    • /
    • v.22 no.8
    • /
    • pp.369-374
    • /
    • 2016
  • The growing demand of virtual computing and storage resources in the cloud computing environment has led to deduplication of storage system for effective reduction and utilization of storage space. In particular, large reduction in the storage space is made possible by preventing data with identical content as the virtual desktop images from being stored on the virtual desktop infrastructure. However, in order to provide reliable support of virtual desktop services, the storage system must address a variety of workloads by virtual desktop, such as performance overhead due to deduplication, periodic data I/O storms and frequent random I/O operations. In this paper, we designed and implemented a clustered file system to support virtual desktop and storage services in cloud computing environment. The proposed clustered file system provides low storage consumption by means of inline deduplication on virtual desktop images. In addition, it reduces performance overhead by deduplication process in the data server and not the virtual host on which virtual desktops are running.

Minimum Bandwidth Regenerating Codes Based on Cyclic VFR Codes

  • Wang, Jing;Wang, Shuxia;Wang, Tiantian;Zhang, Xuefei
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.7
    • /
    • pp.3583-3598
    • /
    • 2019
  • In order to improve the reliability and repair efficiency of distributed storage systems, minimum bandwidth regenerating (MBR) codes based on cyclic variable fractional repetition (VFR) codes are constructed in this thesis, which can repair failed nodes accurately. Specifically, in order to consider the imbalance of data accessed by the users, cyclic VFR codes are constructed according to that data with different heat degrees are copied in different repetition degrees. Moreover, we divide the storage nodes into groups, and construct MBR codes based on cyclic VFR codes to improve the file download speed. Performance analysis and simulation results show that, the repair locality of a single node failure is always 2 when MBR codes based on cyclic VFR codes are adopted in distributed storage systems, which is obviously superior to the traditional MBR codes. Compared with RS codes and simple regenerating codes, the proposed MBR codes based on cyclic VFR codes have lower repair locality, repair complexity and bandwidth overhead, as well as higher repair efficiency. Moreover, relative to FR codes, the MBR codes based on cyclic VFR codes can be applicable to more storage systems.

Implementation of Optical-based Measuring Instrument for Overhead Contact Wire in Railway (전기철도 전차선로의 광학기반 형상 검측 하드웨어 구현)

  • Park, Young;Cho, Yong-Hyeon;Park, Hyun-June;Kwon, Sam-Young
    • Proceedings of the Korean Institute of Electrical and Electronic Material Engineers Conference
    • /
    • 2008.06a
    • /
    • pp.518-518
    • /
    • 2008
  • We propose an optical-based measuring instrument of catenary system in electric railway. This system was made to utilize line scan camera as inspecting system to measure the stagger and height of overhead contact wire in railway and composed with optical type source and FPGA-based image acquisition system with PCI slot. Vision acquisition software has been used for the application to programming interface for image acquisition, display, and storage with a frequency of sampling. The proposed optical-based measuring instrument to measure the contact wire geometry shows promising on-field applications for online condition motoring. Also, this system can be applied to measure the hight and stagger or other geometry of different type of overhead catenary system.

  • PDF