• Title/Summary/Keyword: I/O data

Search Result 1,264, Processing Time 0.026 seconds

Performance Optimization in GlusterFS on SSDs (SSD 환경 아래에서 GlusterFS 성능 최적화)

  • Kim, Deoksang;Eom, Hyeonsang;Yeom, Heonyoung
    • KIISE Transactions on Computing Practices
    • /
    • v.22 no.2
    • /
    • pp.95-100
    • /
    • 2016
  • In the current era of big data and cloud computing, the amount of data utilized is increasing, and various systems to process this big data rapidly are being developed. A distributed file system is often used to store the data, and glusterFS is one of popular distributed file systems. As computer technology has advanced, NAND flash SSDs (Solid State Drives), which are high performance storage devices, have become cheaper. For this reason, datacenter operators attempt to use SSDs in their systems. They also try to install glusterFS on SSDs. However, since the glusterFS is designed to use HDDs (Hard Disk Drives), when SSDs are used instead of HDDs, the performance is degraded due to structural problems. The problems include the use of I/O-cache, Read-ahead, and Write-behind Translators. By removing these features that do not fit SSDs which are advantageous for random I/O, we have achieved performance improvements, by up to 255% in the case of 4KB random reads, and by up to 50% in the case of 64KB random reads.

NMR assignments including HMBC and 1D-TOCSY data of Astragaloside I, II and Isoastragaloside I from the Roots of Astragalus membranaceus (황기뿌리에서 분리한 Astragaloside I, II 및 Isoastragaloside I의 HMBC와 1D-TOCSY data를 포함한 nmr assignments)

  • Park, Jin-Seo;Kim, Chung-Sook;Kim, Jong-Moon;Kim, Jin-Sook
    • Korean Journal of Pharmacognosy
    • /
    • v.31 no.1
    • /
    • pp.34-38
    • /
    • 2000
  • Three compounds were isolated from the roots of Astragalus membranaceus (Leguminosae). On the basis of spectroscopic evidences, the structures were characterized as $3-0-{\beta}-D-xylopyranosyl-(2',3'-O-diacetyl)-6-0-{\beta}-D-glucopyranosyl-3{\beta},6{\alpha},16{\beta},25-tetrahydroxy-20(R)$,24(S)-epoxy-cycloartane(Astragaloside I), $3-0-{\beta}-D-xylopyranosyl-(2'-O-acetyl)-6-0-{\beta}-D-glucopyranosyl-3{\beta},6{\alpha},16{\beta},25-tetrahydroxy-20(R)$,24(S)-epoxy-cycloartane(Astragaloside II), $3-0-{\beta}-D-xylopyranosyl-(2',4'-O-diacetyl)-6-0-{\beta}-D-glucopyranosyl-3{\beta},6{\alpha},16{\beta},25-tetrahydroxy-20(R)$,24(S)-epoxycycloartane(Isoastragaloside I). Full data of NMR including HMBC and 1D-TOCSY experiment of these compounds were reported for the first time.

  • PDF

Implementation of Light-weight I/O Stack for NVMe-over-Fabrics

  • Ahn, Sungyong
    • International journal of advanced smart convergence
    • /
    • v.9 no.3
    • /
    • pp.253-259
    • /
    • 2020
  • Most of today's large-scale cloud systems and enterprise data centers are distributing resources to improve scalability and resource utilization. NVMe-over-Fabric protocol allows submitting NVMe commands to a remote NVMe SSD through RDMA (Remote Direct Memory Access) network. It is attracting attention recently because it is possible to construct a disaggregation storage system with low latency through the protocol. However, the current I/O stack of NVMe-over-Fabric has an inefficient structure for maintaining compatibility with the traditional I/O stack. Therefore, in this paper, we propose a new mechanism to reduce I/O latency and CPU overhead by modifying I/O path of NVMe-over-Fabric to pass through legacy block layer. According to the performance evaluation results, the proposed mechanism is able to reduce the I/O latency and CPU overhead by up to 22% and 24% compared to the existing NVMe-over-Fabrics protocol, respectively.

An Analysis of Utilization on Virtualized Computing Resource for Hadoop and HBase based Big Data Processing Applications (Hadoop과 HBase 기반의 빅 데이터 처리 응용을 위한 가상 컴퓨팅 자원 이용률 분석)

  • Cho, Nayun;Ku, Mino;Kim, Baul;Xuhua, Rui;Min, Dugki
    • Journal of Information Technology and Architecture
    • /
    • v.11 no.4
    • /
    • pp.449-462
    • /
    • 2014
  • In big data era, there are a number of considerable parts in processing systems for capturing, storing, and analyzing stored or streaming data. Unlike traditional data handling systems, a big data processing system needs to concern the characteristics (format, velocity, and volume) of being handled data in the system. In this situation, virtualized computing platform is an emerging platform for handling big data effectively, since virtualization technology enables to manage computing resources dynamically and elastically with minimum efforts. In this paper, we analyze resource utilization of virtualized computing resources to discover suitable deployment models in Apache Hadoop and HBase-based big data processing environment. Consequently, Task Tracker service shows high CPU utilization and high Disk I/O overhead during MapReduce phases. Moreover, HRegion service indicates high network resource consumption for transfer the traffic data from DataNode to Task Tracker. DataNode shows high memory resource utilization and Disk I/O overhead for reading stored data.

Recursive Bus-Invert Coding for Low-Power I/O (저전력 입출력을 위한 반복적인 버스반전 부호화)

  • 정덕기;손윤식정정화
    • Proceedings of the IEEK Conference
    • /
    • 1998.10a
    • /
    • pp.1081-1084
    • /
    • 1998
  • In this paper, we propose the bus coding technique for low power consumption. For CMOS circuit most power is dissipated as dynamic power for charging and discharging node capacitances.Though the I/O and bus are likely to have the very large capacitances associated with them and dissipate much of the power dissipated by an IC, they have little beenthe special target for power reduction. The conventional Bus-Invert coding method can't decrease the peak power dissipation by 50% because the additional invert signal line can invoke a transition at the time when Bus-Invert coding isn't used to code original bus data. The proposed technique always constraints the Hamming distance between data transferred sequentially to be below the half of the bus width, and thus decrease the I/O peak power dissipation and the I/O average power dissipation.

  • PDF

The Design and Implementation of the Multimedia End-to-End Server I/O System based on Linux (멀티미디어 End-to-End 서버용 리눅스 기반 I/O 시스템 설계 및 구현)

  • Nam, Sang-Jun;Lee, Byeong-Rae;Park, Nam-Seop;Lee, Yun-Jeong;Kim, Tae-Yun
    • The KIPS Transactions:PartA
    • /
    • v.8A no.4
    • /
    • pp.311-318
    • /
    • 2001
  • In recent years, users\` demands for multimedia service are increasing because of a diffusion of internet. Server systems, however, offer inefficient multimedia data service to users. Multimedia applications often transfer the same data between shared devices at very high rates, and therefore require an efficient I/O subsystem. Data copying and context switching have long been identified as sources of I/O inefficiency. Therefore we propose the new Multimedia Stream System Call (MSSC) mechanism, which is inserted into a Linux kernel: The MSSC mechanism operates in kernel domain with RTP (Real-time Transport Protocol). We present measurements indicating that use of our techniques resulted in a 12.5%∼14% gain in throughput as compared with a conventional Linux system.

  • PDF

Improving the I/O Performance of Disk-Based Graph Engine by Graph Ordering (디스크 기반 그래프 엔진의 입출력 성능 향상을 위한 그래프 오더링)

  • Lim, Keunhak;Kim, Junghyun;Lee, Eunjae;Seo, Jiwon
    • KIISE Transactions on Computing Practices
    • /
    • v.24 no.1
    • /
    • pp.40-45
    • /
    • 2018
  • With the advent of big data and social networks, large-scale graph processing becomes popular research topic. Recently, an optimization technique called Gorder has been proposed to improve the performance of in-memory graph processing. This technique improves performance by optimizing the graph layout on memory to have better cache locality. However, since it is designed for in-memory graph processing systems, the technique is not suitable for disk-based graph engines; also the cost for applying the technique is significantly high. To solve the problem, we propose a new graph ordering called I/O Order. I/O Order considers the characteristics of I/O accesses for SSDs and HDDs to improve the performance of disk-based graph engine. In addition, the algorithmic complexity of I/O Order is simple compared to Gorder, hence it is cheaper to apply I/O Ordering. I/O order reduces the cost of pre-processing up to 9.6 times compared to that of Gorder's, still its performance is 2 times higher compared to the Random in low-locality graph algorithms.

Analysis of Flood Control Capacity of Agricultural Reservoir Based on SSP Climate Change Scenario (SSP 기후변화 시나리오에 따른 농업용 저수지 홍수조절능력 분석)

  • Kim, Jihye;Kwak, Jihye;Hwang, Soonho;Jun, Sang Min;Lee, Sunghack;Lee, Jae Nam;Kang, Moon Seong
    • Journal of The Korean Society of Agricultural Engineers
    • /
    • v.63 no.5
    • /
    • pp.49-62
    • /
    • 2021
  • The objective of this study was to evaluate the flood control capacity of the agricultural reservoir based on state-of-the-art climate change scenario - SSP (Shared Socioeconomic Pathways). 18 agricultural reservoirs were selected as the study sites, and future rainfall data based on SSP scenario provided by CMIP6 (Coupled Model Intercomparison Project 6) was applied to analyze the impact of climate change. The frequency analysis module, the rainfall-runoff module, the reservoir operation module, and their linkage system were built and applied to simulate probable rainfall, maximum inflow, maximum outflow, and maximum water level of the reservoirs. And the maximum values were compared with the design values, such as design flood of reservoirs, design flood of direct downstream, and top of dam elevation, respectively. According to whether or not the maximum values exceed each design value, cases were divided into eight categories; I-O-H, I-O, I-H, I, O-H, O, H, X. Probable rainfall (200-yr frequency, 12-h duration) for observed data (1973~2020) was a maximum of 445.2 mm and increased to 619.1~1,359.7 mm in the future (2011~2100). For the present, 61.1% of the reservoirs corresponded to I-O, which means the reservoirs have sufficient capacity to discharge large inflow; however, there is a risk of overflowing downstream due to excessive outflow. For the future, six reservoirs (Idong, Baekgok, Yedang, Tapjung, Naju, Jangsung) were changed from I-O to I-O-H, which means inflow increases beyond the discharge capacity due to climate change, and there is a risk of collapse due to dam overflow.

The Accessibility Improvement of Grid Data I/O based on GASS (GASS 기반 그리드 데이터 I/O의 접근성 개선에 관한 연구)

  • Kwon, Oh-Kyoung;Park, Hyoung-Woo;Lee, Sang-San
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2002.11a
    • /
    • pp.147-150
    • /
    • 2002
  • 인터넷 기반의 컴퓨팅 환경이 발전함에 따라 자원과 정보의 공유를 위한 그리드 컴퓨팅이 나타나게 되었다. 그리드 컴퓨팅에서는 그리드 응용 수행시 프로그램과 데이터간의 위치가 다르고 분산되어 있는 경우가 많다. 현재 그리드 미들웨어 시장 표준인 글로버스 툴킷(Globus Toolkit$^{TM}$)에서 사용하는 GASS와 같은 그리드 I/O 시스템들이 원격에 있는 관련 데이터들을 로컬 시스템의 데이터에 접근하는 것처럼 처리하는 것을 지원한다. 그러나 GASS(Global Access to Secondary Storage)에서는 원격지의 파일 시스템의 접근시 URL을 이용하는데, 그리드 환경에서의 I/O을 위한 파일 시스템은 복수의 파일 형태로 분산되어있기 때문에 하나의 그리드 응용을 수행할 때 URL 을 동적으로 바꿔저야 하는 문제점을 갖고 있다. 본 논문에서는 URL 보다 상위 개념의 주소 체계인 URI 을 도입하고 이를 RSL과 연계하여 분산 환경의 그리드 I/O에서도 간을 주소 형태로 I/O 처리가 가능하는 등 효율적인 그리드 I/O를 위한 GASS 개선 방안을 제시 한다.

  • PDF

Development of Point of Production/Manufacturing Execution System to Manage Real-time Plant Floor Data (제품 실명제를 위한 POP/MES 시스템의 개발)

  • Gwon, Yeong-Do;Jo, Chung-Rae;Jeon, Hyeong-Deok
    • 연구논문집
    • /
    • s.27
    • /
    • pp.167-174
    • /
    • 1997
  • Point of Production/Manufacturing Execution Systems are an essential component of operations in today's competitive business environments, which require greater production efficiency and effectiveness. POP/MES focuses on the valuing-adding processes, helping to reduce manufacturing cycle time, improve product quality, reduce WIP, reduce or eliminate paperwork between shifts, reduce lead time and empowering plant operations staff. In this paper, we implement POP/MES to manage real-time plant floor data which is gathered by I/O server into database management system. I/O server is a software allows data exchange between factory real-time database and several hardware devices such as PLC, DCS, robot and sensor through ethernet TCP/IP protocol.

  • PDF