• Title/Summary/Keyword: Large-storage I/O Speed

Search Result 12, Processing Time 0.027 seconds

Large Storage Performance and Optimization Study using blockwrite (blockwrite를 이용한 대형 스토리지 성능 측정 및 최적화 연구)

  • Kim, Hyo-Ryoung;Song, Min-Gyu;Kang, Yong-Woo
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.16 no.6
    • /
    • pp.1145-1152
    • /
    • 2021
  • In order to optimize the performance of 1.4P large storage, the characteristics of each chunk mode were investigated, and the chunk 512K mode was selected in terms of I/O speed. NVME storage system was configured and used to measure data server performance of large storage. By measuring the change in throughput according to the number of threads of the 1.4P large storage, the characteristics of the large storage system were identified, and it was confirmed that the performance was up to 133Gbps with a block size of 32KB. As a result of data transmission/reception experiment using globus-url-copy of GridFTP, it was found that this large storage has a throughput of 33Gbps.

Development and Application of HDD I/O Measurement Utility Blockwrite (하드디스크 데이터 I/O 속도 측정용 유틸리티 blockwrite 개발과 응용)

  • Kim, Hyo-Ryoung;Song, Min-Gyu
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.15 no.6
    • /
    • pp.1151-1158
    • /
    • 2020
  • In order to investigate the speed profile of data I/O of HDD, we have developed an utility program. The application to HDD reveals the detail properties of the speed profile of HDD and the relation between the cylinder structure of HDD and the velocity profile. For the extent application, the experiment of the large volume storage was performed, and the profile of SSD media, which is known as the new rapid media, was measured. The new M.2 NVME SSD, which has the ability of over 10Gbps, we can compare the velocities between cp under linux O/S and the utility, and shows that the performance of the utility can be reliable.

Implementation of Hardware RAID and LVM-based Large Volume Storage on Global Data Center System of International GNSS Service

  • Lee, Dae-Kyu;Cho, Sung-Ki;Park, Jong-Uk;Park, Pil-Ho
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2005.06a
    • /
    • pp.1553-1557
    • /
    • 2005
  • High performance and reliability of the storage system to handle a very large amount of data has been become very important. Many techniques have been applied on the various application systems to establish very large capacity storage that satisfy the requirement of high I/O speed and physical or logical failure protection. We applied RAID and LVM to construct a storage system for the global data center which needs a very reliable large capacity storage system. The storage system is successfully established and equipped on the latest Linux application server.

  • PDF

Cross Compressed Replication Scheme for Large-Volume Column Storages (대용량 컬럼 저장소를 위한 교차 압축 이중화 기법)

  • Byun, Siwoo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.14 no.5
    • /
    • pp.2449-2456
    • /
    • 2013
  • The column-oriented database storage is a very advanced model for large-volume data analysis systems because of its superior I/O performance. Traditional data storages exploit row-oriented storage where the attributes of a record are placed contiguously in hard disk for fast write operations. However, for search-mostly datawarehouse systems, column-oriented storage has become a more proper model because of its superior read performance. Recently, solid state drive using MLC flash memory is largely recognized as the preferred storage media for high-speed data analysis systems. In this paper, we introduce fast column-oriented data storage model and then propose a new storage management scheme using a cross compressed replication for the high-speed column-oriented datawarehouse system. Our storage management scheme which is based on two MLC SSD achieves superior performance and reliability by the cross replication of the uncompressed segment and the compressed segment under high workloads of CPU and I/O. Based on the results of the performance evaluation, we conclude that our storage management scheme outperforms the traditional scheme in the respect of update throughput and response time of the column segments.

Column-aware Polarization Scheme for High-Speed Database Systems (고속 데이터베이스 시스템을 위한 컬럼-인지 양분화 기법)

  • Byun, Si-Woo
    • Journal of Internet Computing and Services
    • /
    • v.13 no.3
    • /
    • pp.83-91
    • /
    • 2012
  • Recently, column-oriented storage has become a progressive model for high-speed database systems because of its superior I/O performance. In this paper, we analysis traditional raw-oriented storage model and then propose a new column-aware storage management model using flash memory drive and assist drive to improve the effective performance of the high-speed column-oriented database system. Our storage management scheme called column-aware polarization improves the performance of update operation by dividing and compressing table columns into active-columns or inactive-columns, and balancing congested update operations using a assist drive in high workload periods. The results obtained from experimental tests show that our scheme improves the update throughput of column-oriented storage by 19 percent, and the response time by up to 49 percent.

Reliable and Flexible Storage System for NAS Environments (NAS를 위한 신뢰성과 확장성이 있는 저장 장치 시스템)

  • Lee, Tae-Keun;Kang, Yong-Hyeog;Eom, Young-Ik
    • The KIPS Transactions:PartD
    • /
    • v.10D no.7
    • /
    • pp.1085-1092
    • /
    • 2003
  • The rapid growth of high speed multimedia services demands the large capacity of the server's storage that maintains the multimedia contents. Among the server's devices, the low I/O speed and physical or logical failure of storage device decrease the total performance of system. The continuous increase of multimedia contents require the flexibility of storage capacity. In order to solve the these problems, we propose the uses of software RAID and LVM techniques that provide the performance improvement and reliability of storage device and the flexibility of storage device respectively. In the LINUX 2.4 kernel, we implemented the reliable and flexible storage device and evaluated the performance of it.

Performance Evaluation of SSD-Index Maintenance Schemes in IR Applications

  • Jin, Du-Seok;Jung, Hoe-Kyung
    • Journal of information and communication convergence engineering
    • /
    • v.8 no.4
    • /
    • pp.377-382
    • /
    • 2010
  • With the advent of flash memory based new storage device (SSD), there is considerable interest within the computer industry in using flash memory based storage devices for many different types of application. The dynamic index structure of large text collections has been a primary issue in the Information Retrieval Applications among them. Previous studies have proven the three approaches to be effective: In- Place, merge-based index structure and a combination of both. The above-mentioned strategies have been researched with the traditional storage device (HDD) which has a constraint on how keep the contiguity of dynamic data. However, in case of the new storage device, we don' have any constraint contiguity problems due to its low access latency time. But, although the new storage device has superiority such as low access latency and improved I/O throughput speeds, it is still not well suited for traditional dynamic index structures because of the poor random write throughput in practical systems. Therefore, using the experimental performance evaluation of various index maintenance schemes on the new storage device, we propose an efficient index structure for new storage device that improves significantly the index maintenance speed without degradation of query performance.

Analysis of Disk Array Architecture as a Storage Server of a Small-Sacle VOD Server (소규모 VOD 시스템의 저장 서버로서 디스크 배열 구조의 분석)

  • Go, Jeong-Guk;Kim, Gil-Yong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.4 no.3
    • /
    • pp.811-820
    • /
    • 1997
  • Disk arrays are using to enhance data trandfer rate and I/O performance in multimedia applications which need a high-performance storage device with large storage capacity and high-speed network.As performance varies with configuration and data layout scheme,disk array characteristic variables must be approrpriately deter-mined in desibning disk array archetecture for a speciffic applicatoin. In this paper,in order to design a disk array architecturte as a storage server of a small-scale VOD system,we evaluate performance of a disk array to chose the number of disks in the array,disk array cinfiguration,a degree of declustering for a given data block size of continous media file system and I/D request size through simulation.Simulation result shows that RAID level 5 with 5 disks ios a suitable candidate for the disk array architecture which privides MPEG-2 files with a rate of 6 Mbps,Moreover,we whow that stripe unit is 64 KB and a layout scheme is contigous placement.

  • PDF

Shadow Recovery for Column-based Databases (컬럼-기반 데이터베이스를 위한 그림자 복구)

  • Byun, Si-Woo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.16 no.4
    • /
    • pp.2784-2790
    • /
    • 2015
  • The column-oriented database storage is a very advanced model for large-volume data transactions because of its superior I/O performance. Traditional data storages exploit row-oriented storage where the attributes of a record are placed contiguously in hard disk for fast write operations. However, for search-mostly data warehouse systems, column-oriented storage has become a more proper model because of its superior read performance. Recently, solid state drive using flash memory is largely recognized as the preferred storage media for high-speed data analysis systems. In this research, we propose a new transaction recovery scheme for a column-oriented database environment which is based on a flash media file system. We improved traditional shadow paging schemes by reusing old data pages which are supposed to be invalidated in the course of writing a new data page in the flash file system environment. In order to reuse these data pages, we exploit reused shadow list structure in our column-oriented shadow recovery(CoSR) scheme. CoSR scheme minimizes the additional storage overhead for keeping shadow pages and minimizes the I/O performance degradation caused by column data compression of traditional recovery schemes. Based on the results of the performance evaluation, we conclude that CoSR outperforms the traditional schemes by 17%.

Study on the Photoelectric Composite Cable for Hybrid Interconnection Implementation (Hybrid 인터커넥션 구현을 위한 광전 복합케이블 제작에 관한 연구)

  • Kim, Jae-Yeol;You, Kwan-Jong;Park, Ryeok
    • Journal of the Korean Society of Manufacturing Process Engineers
    • /
    • v.16 no.3
    • /
    • pp.138-145
    • /
    • 2017
  • With the increasing use of smart electronic devices, the size of the related I/O interface market is increasing rapidly. Demand is also growing for the continuous increase of data and video signals-such as faster data processing speed and data storage capacity-in the smart electronic device input/output interface market. Currently, the POF hybrid cable used in the smart electronic device input / output interface market cannot transmit over a long distance because the optical loss is too large, and the GOF hybrid cable is both vulnerable to bending and other sudden outside changes, and expensive. Therefore, in this study, the design and fabrication of a GOF hybrid cable and fiber guide were carried out in order to develop a cable which can easily withstand external impact, has low optical losses, and meets the demand for continuous data and video signal increase in the smart electronic device input / output interface market.