• Title/Summary/Keyword: log storage device

Search Result 19, Processing Time 0.024 seconds

Performance Analysis of Flash Memory SSD with Non-volatile Cache for Log Storage (비휘발성 캐시를 사용하는 플래시 메모리 SSD의 데이터베이스 로깅 성능 분석)

  • Hong, Dae-Yong;Oh, Gi-Hwan;Kang, Woon-Hak;Lee, Sang-Won
    • Journal of KIISE
    • /
    • v.42 no.1
    • /
    • pp.107-113
    • /
    • 2015
  • In a database system, updates on pages that are made by a transaction should be stored in a secondary storage before the commit is complete. Generic secondary storages have volatile DRAM caches to hide long latency for non-volatile media. However, as logs that are only written to the volatile DRAM cache don't ensure durability, logging latency cannot be hidden. Recently, a flash SSD with capacitor-backed DRAM cache was developed to overcome the shortcoming. Storage devices, like those with a non-volatile cache, will increase transaction throughput because transactions can commit as soon as the logs reach the cache. In this paper, we analyzed performance in terms of transaction throughput when the SSD with capacitor-backed DRAM cache was used as log storage. The transaction throughput can be improved over three times, by committing right after storing the logs to the DRAM cache, rather than to a secondary storage device. Also, we showed that it could acquire over 73% of the ideal logging performance with proper tuning.

A Design and Implementation for a Reliable Data Storage in a Digital Tachograph (디지털 자동차운행기록계에서 안정적인 데이터 저장을 위한 설계 및 구현)

  • Baek, Sung Hoon;Son, Myunghee
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.1 no.2
    • /
    • pp.71-78
    • /
    • 2012
  • The digital tachograph is a device that automatically records speed and distance of a vehicle, together with the driver's activity and vehicle status at an accident. It records vehicle speed, break status, acceleration, engine RPM, longitude and latitude of GPS, accumulated distance, and so on. European Commission regulation made digital tachographs mandatory for all trucks from 2005. Republic of Korea made digital tachographs mandatory for all new business vehicles from 2011 and is widening the range of vehicles that must install digital tachographs year by year. This device is used to analyze driver's daily driving information and car accidents. Under a car accident that makes the device reliability unpredictable, it is very important to store driving information with maximum reliability for its original mission. We designed and implemented a practical digital tachograph. This paper presents a storage scheme that consists of a first storage device with small capacity at a high reliability and a second storage device with large capacity at a low cost in order to reliably records data with a hardware at a low cost. The first storage device records data in a SLC NAND flash memory in a log-structured style. We present a reverse partial scan that overcomes the slow scan time of log-structured storages at the boot stage. The scheme reduced the scan time of the first storage device by 1/50. In addition, our design includes a scheme that fast stores data at a moment of accident by 1/20 of data transfer time of a normal method.

Workload-Driven Adaptive Log Block Allocation for Efficient Flash Memory Management (효율적 플래시 메모리 관리를 위한 워크로드 기반의 적응적 로그 블록 할당 기법)

  • Koo, Duck-Hoi;Shin, Dong-Kun
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.37 no.2
    • /
    • pp.90-102
    • /
    • 2010
  • Flash memory has been widely used as an important storage device for consumer electronics. For the flash memory-based storage systems, FTL (Flash Translation Layer) is used to handle the mapping between a logical page address and a physical page address. Especially, log buffer-based FTLs provide a good performance with small-sized mapping information. In designing the log buffer-based FTL, one important factor is to determine the mapping structure between data blocks and log blocks, called associativity. While previous works use static associativity fixed at the design time, we propose a new log block mapping scheme which adjusts associativity based on the run-time workload. Our proposed scheme improves the I/O performance about 5~16% compared to the static scheme by adjusting the associativity to provide the best performance.

Metadata Log Management for Full Stripe Parity in Flash Storage Systems (플래시 저장 시스템의 Full Stripe Parity를 위한 메타데이터 로그 관리 방법)

  • Lim, Seung-Ho
    • The Journal of Korean Institute of Information Technology
    • /
    • v.17 no.11
    • /
    • pp.17-26
    • /
    • 2019
  • RAID-5 technology is one of the choice for flash storage device to enhance its reliability. However, RAID-5 has inherent parity update overhead, especially, parity overhead for partial stripe write is one of the crucial issues for flash-based RAID-5 technologies. In this paper, we design efficient parity log architecture for RAID-5 to eliminate runtime partial parity overhead. During runtime, partial parity is retained in buffer memory until full stripe write completed, and the parity is written with full strip write. In addition, parity log is maintained in memory until whole the stripe group is used for data write. With this parity log, partial parity can be recovered from the power loss. In the experiments, the parity log method can eliminate partial parity writes overhead with a little parity log writes. Hence it can reduce write amplification at the same reliability.

Flash-Conscious Storage Management Method for DBMS using Dynamic Log Page Allocation (동적 로그 페이지 할당을 이용한 플래시-고려 DBMS의 스토리지 관리 기법)

  • Song, Seok-Il;Khil, Ki-Jeong;Choi, Kil-Seong
    • Journal of Advanced Navigation Technology
    • /
    • v.14 no.5
    • /
    • pp.767-774
    • /
    • 2010
  • Due to advantages of NAND flash memory such as non-volatility, low access latency, low energy consumption, light weight, small size and shock resistance, it has become a better alternative over traditional magnetic disk drives, and has been widely used. Traditional DBMSs including mobile DBMSs may run on flash memory without any modification by using Flash Translation Layer (FTL), which emulates a random access block device to hide the characteristics of flash memory such as "erase-before-update". However, most existing FTLs are optimized for file systems, not for DBMSs, and traditional DBMSs are not aware of them. Also, traditional DBMSs do not consider the characteristics of flash memory. In this paper, we propose a flash-conscious storage system for DBMSs that utilizes flash memory as a main storage medium, and carefully put the characteristics of flash memory into considerations. The proposed flash-conscious storage system exploits log records to avoid costly update operations. It is shown that the proposed storage system outperforms the state.

Data Consistency-Control Scheme Using a Rollback-Recovery Mechanism for Storage Class Memory (스토리지 클래스 메모리를 위한 롤백-복구 방식의 데이터 일관성 유지 기법)

  • Lee, Hyun Ku;Kim, Junghoon;Kang, Dong Hyun;Eom, Young Ik
    • Journal of KIISE
    • /
    • v.42 no.1
    • /
    • pp.7-14
    • /
    • 2015
  • Storage Class Memory(SCM) has been considered as a next-generation storage device because it has positive advantages to be used both as a memory and storage. However, there are significant problems of data consistency in recently proposed file systems for SCM such as insufficient data consistency or excessive data consistency-control overhead. This paper proposes a novel data consistency-control scheme, which changes the write mode for log data depending on the modified data ratio in a block, using a rollback-recovery scheme instead of the Write Ahead Logging (WAL) scheme. The proposed scheme reduces the log data size and the synchronization cost for data consistency. In order to evaluate the proposed scheme, we implemented our scheme on a Linux 3.10.2-based system and measured its performance. The experimental results show that our scheme enhances the write throughput by 9 times on average when compared to the legacy data consistency control scheme.

Partial Garbage Collection Technique for Improving Write Performance of Log-Structured File Systems (부분 가비지 컬렉션을 이용한 로그 구조 파일시스템의 쓰기 성능 개선)

  • Gwak, Hyunho;Shin, Dongkun
    • Journal of KIISE
    • /
    • v.41 no.12
    • /
    • pp.1026-1034
    • /
    • 2014
  • Recently, flash storages devices have become popular. Log-structured file systems (LFS) are suitable for flash storages since these can provide high write performance by only generating sequential writes to the flash device. However, LFS should perform garbage collections (GC) in order to reclaim obsolete space. Recently, a slack space recycling (SSR) technique was proposed to reduce the GC overhead. However, since SSR generates random writes, write performance can be negatively impacted if the random write performance is significantly lower than sequential write performance of the target device. This paper proposes a partial garbage collection technique that copies only a part of valid blocks in a victim segment in order to increase the size of the contiguous invalid space to be used by SSR. The experiments performed in this study show that the write performance in an SD card improves significantly as a result of the partial GC technique.

A Mobile Flash File System - MJFFS (모바일 플래시 파일 시스템 - MJFFS)

  • 김영관;박현주
    • Journal of Information Technology Applications and Management
    • /
    • v.11 no.2
    • /
    • pp.29-43
    • /
    • 2004
  • As the development of an information technique, gradually, mobile device is going to be miniaturized and operates at high speed. By such the requirements, the devices using a flash memory as a storage media are increasing. The flash memory consumes low power, is a small size, and has a fast access time like the main memory. But the flash memory must erase for recording and the erase cycle is limited. JFFS is a representative filesystem which reflects the characteristics of the flash memory. JFFS to be consisted of LSF structure, writes new data to the flash memory in sequential, which is not related to a file size. Mounting a filesystem or an error recovery is achieved through the sequential approach. Therefore, the mounting delay time is happened according to the file system size. This paper proposes a MJFFS to use a multi-checkpoint information to manage a mass flash file system efficiently. A MJFFS, which improves JFFS, divides a flash memory into the block for suitable to the block device, and stores file information of a checkpoint structure at fixed interval. Therefore mounting and error recovery processing reduce efficiently a number of filesystem access by collecting a smaller checkpoint information than capacity of actual files. A MJFFS will be suitable to a mobile device owing to accomplish fast mounting and error recovery using advantage of log foundation filesystem and overcoming defect of JFFS.

  • PDF

Online Snapshot Method based on Directory and File Change Tracking for Virtual File System (가상파일시스템에서 디렉토리 및 파일 변경 추적에 기반한 온라인 스냅샷 방법)

  • Kim, Jinsu;Song, Seokil;Shin, Jae Ryong
    • The Journal of the Korea Contents Association
    • /
    • v.19 no.5
    • /
    • pp.417-425
    • /
    • 2019
  • Storage snapshot technology allows to preserve data at a specific point in time, and recover and access data at a desired point in time. It is an essential technology for storage protection application. Existing snapshot methods have some problems in that they dependent on storage hardware vendor, file system or virtual block device. In this paper, we propose a new snapshot method for solving the problems and creating snapshots on-line. The proposed snapshot method uses a method of extracting the log records of update operations at the virtual file system layer to enable the snapshot method to operate independently on file systems, virtual block devices, and storage hardwares. In addition, the proposed snapshot mehod creates and manages snapshots for directories and files without interruption to the storage service. Finally, through experiments we measure the snapshot creation time and the performance degradation caused by the snapshot.

Monitoring System of File Outflow through Storage Devices and Printers (저장매체와 프린터를 통한 파일유출 모니터링시스템)

  • Choi Joo-ho;Rhew Sung-yul
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.15 no.4
    • /
    • pp.51-60
    • /
    • 2005
  • The riles or intellectual property on computer systems have increasingly been exposed to such threats that they can be flowed out by internal users or outer attacks through the network. The File Outflow Monitoring System monitors file outflows at server by making the toe when users copy files on client computers into storage devices or print them, The monitoring system filters I/O Request packet by I/O Manager in kernel level if files are flowed out by copying, while it uses Win32 API hooking if printed. As a result, it has exactly made the log and monitored file outflows, which is proved through testing in Windows 2000 and XP.