• Title/Summary/Keyword: File Storage

Search Result 453, Processing Time 0.026 seconds

MBS-LVM: A High-Performance Logical Volume Manager for Memory Bus-Connected Storages over NUMA Servers

  • Lee, Yongseob;Park, Sungyong
    • Journal of Information Processing Systems
    • /
    • v.15 no.1
    • /
    • pp.151-158
    • /
    • 2019
  • With the recent advances of memory technologies, high-performance non-volatile memories such as non-volatile dual in-line memory module (NVDIMM) have begun to be used as an addition or an alternative to server-side storages. When these memory bus-connected storages (MBSs) are installed over non-uniform memory access (NUMA) servers, the distance between NUMA nodes and MBSs is one of the crucial factors that influence file processing performance, because the access latency of a NUMA system varies depending on its distance from the NUMA nodes. This paper presents the design and implementation of a high-performance logical volume manager for MBSs, called MBS-LVM, when multiple MBSs are scattered over a NUMA server. The MBS-LVM consolidates the address space of each MBS into a single global address space and dynamically utilizes storage spaces such that each thread can access an MBS with the lowest latency possible. We implemented the MBS-LVM in the Linux kernel and evaluated its performance by porting it over the tmpfs, a memory-based file system widely used in Linux. The results of the benchmarking show that the write performance of the tmpfs using MBS-LVM has been improved by up to twenty times against the original tmpfs over a NUMA server with four nodes.

Boosting up the Mount Latency of NAND Flash File System using Byte-addressable NVRAM (바이트 접근성을 가지는 비휘발성 메모리 소자를 이용한 낸드 플래시 파일 시스템의 부팅시간 개선 기법)

  • Jeon, Byeong-Gil;Kim, Eun-Ki;Shin, Hyung-Jong;Han, Seok-Hee;Won, Yoo-Jip
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.14 no.3
    • /
    • pp.256-260
    • /
    • 2008
  • This paper describes an improvement of mount-time delay in NAND Flash file systems. To improve file system mount performance, this work configures a hierarchical storage system with byte-addressable NVRAM and NAND Flash memory, and let the meta data of a file system allocated in the NVRAM. Since the meta data are stored in NVRAM supporting data integrity some of the items, which are stored in Spare area and Object Header area of NAND Flash memory to control meta data of NAND Flash file system, could be eliminated. And also, this work eliminates the scanning operation of the Object Header area of previous work FRASH1.0. The scanning operation is definitely required to find out the empty Object Header address for storing the Object Header data and provokes a certain amount of performance loss in file generation and deletion. In this work, an implemented file system, so-called FRASH1.5, is demonstrated, featuring new data structures and new algorithms. The mount time of FRASH1.5 becomes twice as fast as that of the FRASH1.0. The performance in file generation gets improved by about $3{\sim}8%$. In particular, for most large-size files, the FRASH1.5 has 8 times faster mount time than YAFFS, without any performance loss as seen in the file generation.

Implement of MapReduce-based Big Data Processing Scheme for Reducing Big Data Processing Delay Time and Store Data (빅데이터 처리시간 감소와 저장 효율성이 향상을 위한 맵리듀스 기반 빅데이터 처리 기법 구현)

  • Lee, Hyeopgeon;Kim, Young-Woon;Kim, Ki-Young
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.10
    • /
    • pp.13-19
    • /
    • 2018
  • MapReduce, the Hadoop's essential core technology, is most commonly used to process big data based on the Hadoop distributed file system. However, the existing MapReduce-based big data processing techniques have a feature of dividing and storing files in blocks predefined in the Hadoop distributed file system, thus wasting huge infrastructure resources. Therefore, in this paper, we propose an efficient MapReduce-based big data processing scheme. The proposed method enhances the storage efficiency of a big data infrastructure environment by converting and compressing the data to be processed into a data format in advance suitable for processing by MapReduce. In addition, the proposed method solves the problem of the data processing time delay arising from when implementing with focus on the storage efficiency.

Secure Deletion for Flash Memory File System (플래시메모리 파일시스템을 위한 안전한 파일 삭제 기법)

  • Sun, Kyoung-Moon;Choi, Jong-Moo;Lee, Dong-Hee;Noh, Sam-H.
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.13 no.6
    • /
    • pp.422-426
    • /
    • 2007
  • Personal mobile devices equipped with non-volatile storage such as MP3 player, PMP, cellular phone, and USB memory require safety for the stored data on the devices. One of the safety requirements is secure deletion, which is removing stored data completely so that the data can not be restored illegally. In this paper, we study how to design the secure deletion on Flash memory, commonly used as storage media for mobile devices. We consider two possible secure deletion policy, named zero-overwrite and garbage-collection respectively, and analyze how each policy affects the performance of Flash memory file systems. Then, we propose an adaptive file deletion scheme that exploits the merits of the two possible policies. Specifically, the proposed scheme applies the zero-overwrite policy for small files, whereas it employs the garbage-collection policy for large files. Real implementation experiments show that the scheme is not only secure but also efficient.

Web-based Medical Information System supporting DICOM Specification (DICOM 표준을 지원하는 웹 기반 의료 정보 시스템)

  • Kwon, Gi-Beom;Kim, Il-Kon
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.7 no.4
    • /
    • pp.317-323
    • /
    • 2001
  • DICOM(Digital Imaging and Communications in MediCine), standard of medical image operation, present the methods for communications and Storage of Medical Image. medical image acquired from patient in hospital made DICOM files. this paper purposes design and implementation methodologies of a web-based medical information system that consists of DICOM (Digital Imaging and Communications in Medicine) databases and functional components of a web server in order to support the access of medical information with Intemet web browser. we store the patient and image information to database using reading the group and element oJ DICOM file. we made file transfer module by implementing DICOM Store service, in result, we can transfer DICOM file to IF based host or computer. We compose web component of communications and Storage service, user be used DICOM Service by web Browser.

  • PDF

Design and Implementation of Buffer Cache for EXT3NS File System (EXT3NS 파일 시스템을 위한 버퍼 캐시의 설계 및 구현)

  • Sohn, Sung-Hoon;Jung, Sung-Wook
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.10 no.12
    • /
    • pp.2202-2211
    • /
    • 2006
  • EXT3NS is a special-purpose file system for large scale multimedia streaming servers. It is built on top of streaming acceleration hardware device called Network-Storage card. The EXT3NS file system significantly improves streaming performance by eliminating memory-to-memory copy operations, i.e. sending video/audio from disk directly to network interface with no main memory buffering. In this paper, we design and implement a buffer cache mechanism, called PMEMCACHE, for EXT3NS file system. We also propose a buffer cache replacement method called ONS for the buffer cache mechanism. The ONS algorithm outperforms other existing buffer replacement algorithms in distributed multimedia streaming environment. In EXT3NS with PMEMCACHE, operation is 33MB/sec and random read operation is 2.4MB/sec. Also, the buffer replacement ONS algorithm shows better performance by 600KB/sec than other buffer cache replacement policies. As a result PMEMCACHE and an ONS can greatly improve the performance of multimedia steaming server which should supportmultiple client requests at the same time.

Generation of Spatial Adjacency Map and Contents File Format for Ultra Wide Viewing Service (울트라 와이드 뷰잉 서비스를 위한 공간 유사도 맵 생성 및 울트라 와이드 뷰잉 콘텐츠 저장 방법)

  • Lee, Euisang;Kang, Dongjin;Kim, Kyuheon
    • Journal of Broadcast Engineering
    • /
    • v.22 no.4
    • /
    • pp.473-483
    • /
    • 2017
  • Since the advent of 3D and UHD contents, demand for high-quality panoramic images has been increasing. The UWV(Ultra-Wide Viewing) service uses a wider viewing angle than conventional panoramas to provide a lively experience for users and enhance their understanding of the event. In this paper, we propose a spatial adjacency map generation method and an UWV file storage format technology to provide UWV service. The spatial adjacency map measures the similarity between images and generate the position information of the images based on similarity. And the stitching time of the image can be shortened through the generated position information. Through the spatial adjacency map, we generate the large screen content quickly. The UWV file format which is based on ISOBMFF process spatial adjacency map and videos and support random access. In this paper, we design the UWV player to verify the spatial adjacency map and UWV file format and show the result of experiments.

BubbleDoc: Document Forgery and Tamper Detection through the Agent-Free File System-Awareness in Cloud Environment (BubbleDoc: 클라우드 환경에서의 agent-free 파일시스템 분석을 통한 문서 위/변조 탐지)

  • Jeon, Woo-Jin;Hong, Dowon;Park, Ki-Woong
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.28 no.2
    • /
    • pp.429-436
    • /
    • 2018
  • Electronic documents are efficient to be created and managed, but they are liable to lose their originality because copies are created during distribution and delivery. For this reason, various security technologies for electronic documents have been applied. However, most security technologies currently used are for document management such as file access privilege control, file version and history management, and therefore can not be used in environments where authenticity is absolutely required, such as confidential documents. In this paper, we propose a method to detect document forgery and tampering through analysis of file system without installing an agent inside the instance operating system in cloud computing environment. BubbleDoc monitors the minimum amount of virtual volume storage in an instance, so it can efficiently detect forgery and tampering of documents. Experimental results show that the proposed technique has 0.16% disk read operation overhead when it is set to 1,000ms cycle for monitoring for document falsification and modulation detection.

A Segment Space Recycling Scheme for Optimizing Write Performance of LFS (LFS의 쓰기 성능 최적화를 위한 세그먼트 공간 재활용 기법)

  • Oh, Yong-Seok;Kim, Eun-Sam;Choi, Jong-Moo;Lee, Dong-Hee;Noh, Sam-H.
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.15 no.12
    • /
    • pp.963-967
    • /
    • 2009
  • The Log-structured File System (LFS) collects all modified data into a memory buffer and writes them sequentially to a segment on disk. Therefore, it has the potential to utilize the maximum bandwidth of storage devices where sequential writes are much faster than random writes. However, as disk space is finite, LFS has to conduct cleaning to produce free segments. This cleaning operation is the main reason LFS performance deteriorates when file system utilization is high. To overcome painful cleaning and reduced performance of LFS, we propose the segment space recycling (SSR) scheme that directly writes modified data to invalid areas of the segments and describe the classification method of data and segment to consider locality of reference for optimizing SSR scheme. We implement U-LFS, which employs our segment space recycling scheme in LFS, and experimental results show that SSR scheme increases performance of WOLF by up to 1.9 times in HDD and 1.6 times in SSD when file system utilization is high.

Design and Implementation of File System Using Local Buffer Cache for Digital Convergence Devices (디지털 컨버전스 기기를 위한 지역 버퍼 캐쉬 파일 시스템 설계 및 구현)

  • Jeong, Geun-Jae;Cho, Moon-Haeng;Lee, Cheol-Hoon
    • The Journal of the Korea Contents Association
    • /
    • v.7 no.8
    • /
    • pp.21-30
    • /
    • 2007
  • Due to the growth of embedded systems and the development of semi-conductor and storage devices, digital convergence devises is ever growing. Digital convergence devices are equipments into which various functions such as communication, playing movies and wave files and electronic dictionarys are integrated. Example are portable multimedia players(PMPs), personal digital assistants(PDAs), and smart phones. Therefore, these devices need an efficient file system which manages and controls various types of files. In designing such file systems, the size constraint for small embedded systems as well as performance and compatibility should be taken into account. In this paper, we suggest the partial buffer cache technique. Contrary to the traditional buffer cache, the partial buffer cache is used for only the FAT meta data and write-only data. Simulation results show that we could enhance the write performance more than 30% when the file size is larger than about 100 KBytes.