• Title/Summary/Keyword: File Storage

Search Result 453, Processing Time 0.023 seconds

Study of Optimization through Performance Analysis of Parallel Distributed Filesystem (병렬 분산파일시스템의 성능 분석을 통한 최적화 연구)

  • Yoon, JunWeon;Song, Ui-Sung
    • Journal of Digital Contents Society
    • /
    • v.17 no.5
    • /
    • pp.409-416
    • /
    • 2016
  • Recently, Big Data issue has become a buzzword and universities, industries and research institutes have been efforts to collect, analyze various data enabled. These things includes accumulated data from the past, even if it is not possible to analysis at this present immediately a which has the potential means. And we are obtained a valuable result from the collected a large amount of data via the semantic analysis. The demand for high-performance storage system that can handle large amounts of data required is increasing around the world. In addition, it must provide a distributed parallel file system that stability to multiple users too perform a variety of analyzes at the same time by connecting a large amount of the accumulated data In this study, we identify the I/O bandwidth of the storage system to be considered, and performance of the metadata in order to provide a file system in stability and propose a method for configuring the optimal environment.

An Efficient Recovery Technique using Global Buffer on SAN Environments (SAN 환경에서의 전역 버퍼를 이용한 효율적인 회복 기법)

  • Park, Chun-Seo;Kim, Gyeong-Bae;Lee, Yong-Ju;Park, Seon-Yeong;Sin, Beom-Ju
    • The KIPS Transactions:PartA
    • /
    • v.8A no.4
    • /
    • pp.375-384
    • /
    • 2001
  • The shared disk file systems use a technique known as file system journaling to support recovery of metadata on the SAN(Storage Area Network). In the existing journaling technique, the metadata that is dirtied by one host must be updated to disk space before some hosts access it. The system performance is decreased because the disk access number is increased. In this paper, we describe a new recovery technique using a global buffer to decrease disk I/O. It transmits the dirtied metadata into the other hosts through Fibre Channel network on the SAN instead of disk I/O and supports recovery of a critical data by journaling a data as well as metadata.

  • PDF

Efficient Update Method for Cloud Storage System

  • Khill, Ki-Jeong;Lee, Sang-Min;Kim, Young-Kyun;Shin, Jaeryong;Song, Seokil
    • International Journal of Contents
    • /
    • v.10 no.1
    • /
    • pp.62-67
    • /
    • 2014
  • Usually, cloud storage systems are developed based on DFS (Distributed File System) for scalability and reliability reasons. DFSs are designed to improve throughput than IO response time, and therefore, they are appropriate for batch processing jobs. Recently, cloud storage systems have been used for update intensive applications such as OLTP and so on. However, in DFSs, in-place update operations are not carefully considered. Therefore, when updates are frequent, I/O performance of DFSs are degraded significantly. DFSs with RAID techniques have been proposed to improve their performance and reliability. Their performance degradation caused by frequent update operations can be more significant. In this paper, we propose an in-place update method for DFS RAID exploiting a differential logging technique. The proposed method reduces the I/O costs, network traffic and XOR operation costs for RAID. We demonstrate the efficiency of our proposed in-place update method through various experiments.

Dynamic Load Balancing and Network Adaptive Virtual Storage Service for Mobile Appliances

  • Ong, Ivy;Lim, Hyo-Taek
    • Journal of Information Processing Systems
    • /
    • v.7 no.1
    • /
    • pp.53-62
    • /
    • 2011
  • With the steady growth of mobile technology and applications, demand for more storage in mobile devices has also increased. A lightweight block-level protocol, Internet Advanced Technology Attachment (iATA), has been developed to deliver a cost-effective storage network solution for mobile devices to obtain more storage. This paper seeks to contribute to designing and implementing Load Balancing (LB), Network Monitoring (NM) and Write Replication (WR) modules to improve the protocol's scalability and data availability. LB and NM modules are invoked to collect system resources states and current network status at each associate node (server machine). A dynamic weight factor is calculated based on the collected information and sent to a referral server. The referral server is responsible to analyze and allocate the most ideal node with the least weight to serve the client. With this approach, the client can avoid connecting to a heavily loaded node that may cause delays in subsequent in-band I/O operations. Write replication is applied to the remaining nodes through a WR module by utilizing the Unison file synchronization program. A client initially connected to node IP A for write operations will have no hindrances in executing the relevant read operations at node IP B in new connections. In the worst case scenario of a node crashing, data remain recoverable from other functioning nodes. We have conducted several benchmark tests and our results are evaluated and verified in a later section.

Minimum Bandwidth Regenerating Codes Based on Cyclic VFR Codes

  • Wang, Jing;Wang, Shuxia;Wang, Tiantian;Zhang, Xuefei
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.7
    • /
    • pp.3583-3598
    • /
    • 2019
  • In order to improve the reliability and repair efficiency of distributed storage systems, minimum bandwidth regenerating (MBR) codes based on cyclic variable fractional repetition (VFR) codes are constructed in this thesis, which can repair failed nodes accurately. Specifically, in order to consider the imbalance of data accessed by the users, cyclic VFR codes are constructed according to that data with different heat degrees are copied in different repetition degrees. Moreover, we divide the storage nodes into groups, and construct MBR codes based on cyclic VFR codes to improve the file download speed. Performance analysis and simulation results show that, the repair locality of a single node failure is always 2 when MBR codes based on cyclic VFR codes are adopted in distributed storage systems, which is obviously superior to the traditional MBR codes. Compared with RS codes and simple regenerating codes, the proposed MBR codes based on cyclic VFR codes have lower repair locality, repair complexity and bandwidth overhead, as well as higher repair efficiency. Moreover, relative to FR codes, the MBR codes based on cyclic VFR codes can be applicable to more storage systems.

Randomized Block Size (RBS) Model for Secure Data Storage in Distributed Server

  • Sinha, Keshav;Paul, Partha;Amritanjali, Amritanjali
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.12
    • /
    • pp.4508-4530
    • /
    • 2021
  • Today distributed data storage service are being widely used. However lack of proper means of security makes the user data vulnerable. In this work, we propose a Randomized Block Size (RBS) model for secure data storage in distributed environments. The model work with multifold block sizes encrypted with the Chinese Remainder Theorem-based RSA (C-RSA) technique for end-to-end security of multimedia data. The proposed RBS model has a key generation phase (KGP) for constructing asymmetric keys, and a rand generation phase (RGP) for applying optimal asymmetric encryption padding (OAEP) to the original message. The experimental results obtained with text and image files show that the post encryption file size is not much affected, and data is efficiently encrypted while storing at the distributed storage server (DSS). The parameters such as ciphertext size, encryption time, and throughput have been considered for performance evaluation, whereas statistical analysis like similarity measurement, correlation coefficient, histogram, and entropy analysis uses to check image pixels deviation. The number of pixels change rate (NPCR) and unified averaged changed intensity (UACI) were used to check the strength of the proposed encryption technique. The proposed model is robust with high resilience against eavesdropping, insider attack, and chosen-plaintext attack.

A Study on Analysis of Hidden Areas of Removable Storage Device from a Digital Forensics Point of View (디지털 포렌식 관점에서 이동식 저장매체의 은닉영역 분석 연구)

  • Hong, Pyo-gil;Lee, Dae-sung;Kim, Dohyun
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2021.05a
    • /
    • pp.111-113
    • /
    • 2021
  • USB storage devices, which are represented by removable storage media, are widely used even nowadays when cloud services are common. However, since they are cases where hidden areas are created and exploited in USB storage devices. This research is needed to detect and analyze them from an Anti-forensic point of view. In this paper, we analyze a program that can be exploited as Anti-forensic because it can create a hidden partition and store files there, and the file system created by it from a digital forensic point of view.

  • PDF

Fuzzy Inference for Idle Time Optimization of Hard Disk Drive (퍼지 추론을 이용한 하드디스크드라이브의 유휴시간 최적화)

  • Jun, Jin-Wan;Kim, Gyu-Taek;Lee, Jee-Hyong
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.57 no.3
    • /
    • pp.473-479
    • /
    • 2008
  • Generally, HDDs are widely used as data storage device in office, home and mobile machineries. So, it is used for various applications or tasks, such as file copy, file download, music and movie play etc., in various environment. In spite of this kind of varieties in tasks and environment in which HDDs perform, most commercial HDDs hardly control its operations adaptively to these varieties. Thus, it is preferred to optimize the performance and energy consumption of HDDs according to the task and/or the environment. So, this paper proposes a new fuzzy inference algorithm which adaptively controls HDDs operations and may also easily be implemented as the firmware of HDDs and run in the restricted environment such as embedded systems.

Storage Space Page Grouping in a Distributed Local File System on Desktop Systems to Optimize File Availability (데스크탑 시스템을 이용한 분산 지역 파일 시스템에서의 파일 가용도 최적화를 위한 저장 공간 페이지 배치 기법)

  • 김준우;신현식;장래혁
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2001.10c
    • /
    • pp.856-858
    • /
    • 2001
  • 분산 파일 시스템에서 파일을 복제하는 목적의 하나는 파일의 가용성을 높이는 것이다 서버들로 이루어진 시스템에서 가용도 문제는 주로 오류 상황이 대상이다 반면에 데스크탑으로 구성된 경우 각각이 상대적으로 가용도가 낮고 그 편차도 커서 파일의 가용도를 최적화 하도록 복제본을 저장하는 것이 기본적인 문제가 된다. 파일 단위 복제 기법에서는 파일 수가 참여 시스템 수에 따라 기하급수적으로 증가하고 파일 배치 때마다 가용도 최적화를 수행해야 하는 문제가 있다. 본 논문에서는 저장 공간 단위 복제 기법을 제안한다. 데스크탑 시스템의 저 장롱간을 고정 크기의 페이지들로 나누고 이들을 조합해서 일정한 가용도를 가지는 페이지 그룹을 형성한다. 파일들은 이 페이지 그룹에 저장된다. 페이지의 개수가 파일 수가 아닌 시스템 수에 의존하고, 파일 배치 시점과 무관하게 페이지 그룹을 만들 수 있는 장점이 있다. 최종 가용도를 높이고 연산 시간을 줄이기 위해 ILS(Iterated Local Search )방법을 이용만 탐색을 수행한다.

  • PDF

Image Identifier Based on Linear Component Extraction using Hough Transform (허프변환을 이용한 직선요소 검출 기반 정지영상 인식자)

  • Park, Je-Ho
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.5 no.3
    • /
    • pp.111-117
    • /
    • 2010
  • The easily accessible handheld devices equipped with camera are widely available as common commodities. According to this trend, utilization of images is popular among common users for various purposes resulting in huge amount of images in local or network based storage systems. In this environment, identification of an image with a solid and effective manner is demanded in behalf of safe distribution and efficient management of images. The generated identifiers can be used as a file name in file systems or an index in image databases utilizing the uniqueness of the identifiers. In this paper, we propose a method that generates image identifiers using linear components in images. Some experiments of generation of identifiers are performed, and the results evaluate that the proposed method has feasible effectiveness.