• Title/Summary/Keyword: Distributed Storage

Search Result 701, Processing Time 0.024 seconds

A Development of VPP Platform for the Efficient Utilization of Distributed Renewable Energy Resources (분산 재생에너지의 효율적 활용을 위한 가상발전소(VPP) 플랫폼 개발에 관한 연구)

  • Cho, Young-Hyeok;Baek, Seung-Yup;Choi, Won-Yong;Jeong, Dae-Yul
    • The Journal of Information Systems
    • /
    • v.27 no.2
    • /
    • pp.95-114
    • /
    • 2018
  • Purpose The recent concern over environmental problems such as greenhouse gas emission and fine dust contributes increasing interest in renewable energies. However the intrinsic characteristics of renewable energies, intermittent and stochastic generation, might cause serious problems to the stability and controllability of power grid. Therefore countermeasures such as virtual power plant (VPP) must be prepared in advance of the spread of uncontrollable distributed renewable energy resources to be one of major energy sources. Design/methodology/approach This study deals with the design concept of the VPP platform. we proposed as a technology solution for achieving the stability of power grid by guaranteeing a single power profile combining multiple distributed power sources with ICT. The core characteristics of VPP should be able to participate in the grid operation by responding to operation instructions from the system operator, KPX, as well as the wholesale electricity market. Findings Therefore this study includes energy storage device(ESS) as a controllable component as well as renewable energy resources such as photovoltaic and wind power generation. Based on this configuration, we discussed core element technologies of VPP and protype design of VPP solution platform according to system requirements. In the proposed solution platform, UX design for the integrated control center and brokerage system were included as well as ancillary service function to respond to KPX's operation instruction with utilizing the capability of ESS. In addition, a simulator was suggested to verify the VPP operations.

The Design of Regenerating Codes with a Varying Number of Helper Nodes (다양한 도움 노드의 수를 가지는 재생 부호의 설계)

  • Lee, Hyuk;Lee, Jungwoo
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.41 no.12
    • /
    • pp.1684-1691
    • /
    • 2016
  • Erasure codes have recently been applied to distributed storage systems due to their high storage efficiency. Regenerating codes are a kind of erasure codes, which are optimal in terms of minimum repair bandwidth. An (n,k,d)-regenerating code consists of n storage nodes where a failed node can be recovered with the help of the exactly d numbers of surviving nodes. However, if node failures occur frequently or network connection is unstable, the number of helper nodes that a failed node can contact may be smaller than d. In such cases, regenerating codes cannot repair the failed nodes efficiently since the node repair process of the codes does not work when the number of helper nodes is less than d. In this paper, we propose an operating method of regenerating codes where a failed node can be repaired from ${\bar{d}}$ helper nodes where $$k{\leq_-}{\bar{d}}{\leq_-}d$$.

Study of Optimization through Performance Analysis of Parallel Distributed Filesystem (병렬 분산파일시스템의 성능 분석을 통한 최적화 연구)

  • Yoon, JunWeon;Song, Ui-Sung
    • Journal of Digital Contents Society
    • /
    • v.17 no.5
    • /
    • pp.409-416
    • /
    • 2016
  • Recently, Big Data issue has become a buzzword and universities, industries and research institutes have been efforts to collect, analyze various data enabled. These things includes accumulated data from the past, even if it is not possible to analysis at this present immediately a which has the potential means. And we are obtained a valuable result from the collected a large amount of data via the semantic analysis. The demand for high-performance storage system that can handle large amounts of data required is increasing around the world. In addition, it must provide a distributed parallel file system that stability to multiple users too perform a variety of analyzes at the same time by connecting a large amount of the accumulated data In this study, we identify the I/O bandwidth of the storage system to be considered, and performance of the metadata in order to provide a file system in stability and propose a method for configuring the optimal environment.

Improving Fault Tolerance for High-capacity Shared Distributed File Systems using the Rotational Lease Under Network Partitioning (대용량 공유 분산 화일 시스템에서 망 분할 시 순환 리스를 사용한 고장 감내성 향상)

  • Tak, Byung-Chul;Chung, Yon-Dohn;Kim, Myoung-Ho
    • Journal of KIISE:Databases
    • /
    • v.32 no.6
    • /
    • pp.616-627
    • /
    • 2005
  • In the shared storage file system, systems can directly access the shared storage device through specialized data-only subnetwork unlike in the network attached file server system. In this shared-storage architecture, data consistency is maintained by some designated set of lock servers which use control network to send and receive the lock information. Furthermore, lease mechanism is introduced to cope with the control network failure. But when the control network is partitioned, participating systems can no longer make progress after the lease term expires until the network recovers. This paper addresses this limitation and proposes a method that allows partitioned systems to make progress under the partition of control network. The proposed method works in a manner that each participating system is rotationally given a predefined lease term periodically. It is also shown that the proposed mechanism always preserves data consistency.

An Efficient Implementation of Mobile Raspberry Pi Hadoop Clusters for Robust and Augmented Computing Performance

  • Srinivasan, Kathiravan;Chang, Chuan-Yu;Huang, Chao-Hsi;Chang, Min-Hao;Sharma, Anant;Ankur, Avinash
    • Journal of Information Processing Systems
    • /
    • v.14 no.4
    • /
    • pp.989-1009
    • /
    • 2018
  • Rapid advances in science and technology with exponential development of smart mobile devices, workstations, supercomputers, smart gadgets and network servers has been witnessed over the past few years. The sudden increase in the Internet population and manifold growth in internet speeds has occasioned the generation of an enormous amount of data, now termed 'big data'. Given this scenario, storage of data on local servers or a personal computer is an issue, which can be resolved by utilizing cloud computing. At present, there are several cloud computing service providers available to resolve the big data issues. This paper establishes a framework that builds Hadoop clusters on the new single-board computer (SBC) Mobile Raspberry Pi. Moreover, these clusters offer facilities for storage as well as computing. Besides the fact that the regular data centers require large amounts of energy for operation, they also need cooling equipment and occupy prime real estate. However, this energy consumption scenario and the physical space constraints can be solved by employing a Mobile Raspberry Pi with Hadoop clusters that provides a cost-effective, low-power, high-speed solution along with micro-data center support for big data. Hadoop provides the required modules for the distributed processing of big data by deploying map-reduce programming approaches. In this work, the performance of SBC clusters and a single computer were compared. It can be observed from the experimental data that the SBC clusters exemplify superior performance to a single computer, by around 20%. Furthermore, the cluster processing speed for large volumes of data can be enhanced by escalating the number of SBC nodes. Data storage is accomplished by using a Hadoop Distributed File System (HDFS), which offers more flexibility and greater scalability than a single computer system.

Comparison of Runoff Analysis Between GIS-based Distributed Model and Lumped Model for Flood Forecast of Dam Watershed (댐유역 홍수예측을 위한 GIS기반의 분포형모형과 집중형모형의 유출해석 비교)

  • Park, Jin-Hyeog;Kang, Boo-Sik
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.9 no.3
    • /
    • pp.171-182
    • /
    • 2006
  • In this study, rainfall-runoff analysis was performed for Yongdam watershed($930km^2$) using KOWACO flood analysis model based on Storage Function Method as lumped hydrologic model and Vflo which was developed for real-time flood prediction by University of Oklahoma. The results shows that, the hydrographs of lumped and distributed model with uncalibrated parameters which estimated from physical or experimental relationship show significant biases from observed hydrographs. However, the hydrograph at Cheoncheon site from the distributed model follows the actual hydrograph to an extent that no more calibration is necessary. It encourages that distributed model can have advantages for application in real-time flood forecasting as physically based distributed hydrologic model which can construct event-independent basin parameter group.

  • PDF

Design and Implementation of a Performance Monitoring and Configuration Management Tool for SANtopia (SANtopia를 위한 성능 감시 및 구성 관리 도구의 설계 및 구현)

  • Rim Kee-Wook;Na Yong-Hi;Min Byoung-Joon;Seo Dea-Wha;Shin Bum-Joo
    • Journal of Internet Computing and Services
    • /
    • v.4 no.1
    • /
    • pp.53-65
    • /
    • 2003
  • I/O processing speed of relatively sluggish storage devices incurs overall performance degradation in computer systems. As a solution to improve the situation, SAN(Storage Area Network) has been proposed. In order to utilize a SAN system more effectively, where storage devices are directly connected with a high speed network such as Fibre Channel, a proper management tool is needed. In this paper, we present a design and implementation of a performance monitoring and configuration management tool for the SANtopia system which supports global file sharing in the SAN environment The developed tool is to monitor the performance of the Linux hosts composing the SANtopia system and to manage the configuration of the hosts and storage devices. It also supports GUI(Graphic User Interface) environment using the JAVA programming language.

  • PDF

Development of Climate & Environment Data System for Big Data from Climate Model Simulations (대용량 기후모델자료를 위한 통합관리시스템 구축)

  • Lee, Jae-Hee;Sung, Hyun Min;Won, Sangho;Lee, Johan;Byu, Young-Hwa
    • Atmosphere
    • /
    • v.29 no.1
    • /
    • pp.75-86
    • /
    • 2019
  • In this paper, we introduce a novel Climate & Environment Database System (CEDS). The CEDS is developed by the National Institute of Meteorological Sciences (NIMS) to provide easy and efficient user interfaces and storage management of climate model data, so improves work efficiency. In uploading the data/files, the CEDS provides an option to automatically operate the international standard data conversion (CMORization) and the quality assurance (QA) processes for submission of CMIP6 variable data. This option increases the system performance, removes the user mistakes, and increases the level of reliability as it eliminates user operation for the CMORization and QA processes. The uploaded raw files are saved in a NAS storage and the Cassandra database stores the metadata that will be used for efficient data access and storage management. The Metadata is automatically generated when uploading a file, or by the user inputs. With the Metadata, the CEDS supports effective storage management by categorizing data/files. This effective storage management allows easy and fast data access with a higher level of data reliability when requesting with the simple search words by a novice. Moreover, the CEDS supports parallel and distributed computing for increasing overall system performance and balancing the load. This supports the high level of availability as multiple users can use it at the same time with fast system-response. Additionally, it deduplicates redundant data and reduces storage space.

Implementation and Performance Measuring of Erasure Coding of Distributed File System (분산 파일시스템의 소거 코딩 구현 및 성능 비교)

  • Kim, Cheiyol;Kim, Youngchul;Kim, Dongoh;Kim, Hongyeon;Kim, Youngkyun;Seo, Daewha
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.41 no.11
    • /
    • pp.1515-1527
    • /
    • 2016
  • With the growth of big data, machine learning, and cloud computing, the importance of storage that can store large amounts of unstructured data is growing recently. So the commodity hardware based distributed file systems such as MAHA-FS, GlusterFS, and Ceph file system have received a lot of attention because of their scale-out and low-cost property. For the data fault tolerance, most of these file systems uses replication in the beginning. But as storage size is growing to tens or hundreds of petabytes, the low space efficiency of the replication has been considered as a problem. This paper applied erasure coding data fault tolerance policy to MAHA-FS for high space efficiency and introduces VDelta technique to solve data consistency problem. In this paper, we compares the performance of two file systems, MAHA-FS and GlusterFS. They have different IO processing architecture, the former is server centric and the latter is client centric architecture. We found the erasure coding performance of MAHA-FS is better than GlusterFS.

Application and Policy Direction of Blockchain in Logistics and Distribution Industry (물류 및 유통산업의 블록체인 활용과 정책 방향)

  • Kim, Ki-Heung;Shim, Jae-Hyun
    • The Journal of Industrial Distribution & Business
    • /
    • v.9 no.6
    • /
    • pp.77-85
    • /
    • 2018
  • Purpose - The purpose of this study is to subdivide trade transaction-centered structure in a logistics/distribution industry system to apply blockchain, to establish and resolve with which types of technology, and to provide policy direction of government institution and technology to apply blockchain in this kind of industry. Research design, data, and methodology - This study was conducted with previous researches centered on cases applied in various industry sectors on the basis of blockchain technology. Results - General fields of blockchain application include digital contents distribution, IoT platform, e-Commerce, real-estate transaction, decentralized app. development(storage), certification service, smart contract, P2P network infrastructure, publication/storage of public documents, smart voting, money exchange, payment/settlement, banking security platform, actual asset storage, stock transaction and crowd funding. Blockchain is being applied in various fields home and abroad and its application cases can be explained in the banking industry, public sector, e-Commerce, medical industry, distribution and supply chain management, copyright protection. As examined in the blockchain application cases, it is expected to establish blockchain that can secure safety through distributed ledger in trade transaction because blockchain is established and applied in various sectors of industries home and abroad. Parties concerned of trade transaction can secure visibility even in interrupted specific section when they provide it as a base for distributed ledger application in trade and establish trade transaction model by applying blockchain. In case of interrupted specific section by using distributed ledger, blockchain model of trade transaction needs to be formed to make it possible for parties concerned involved in trade transaction to secure visibility and real-time tracking. Additionally, management should be possible from the time of contract until payment, freight transfer to buyers through land, air and maritime transportation. Conclusions - In order to boost blockchain-based logistics/distribution industry, the government, institutionally, needs to back up adding legal plan of shipping, logistics and distribution, reviewing standardization of electronic switching system and coming up with blockchain-based industrial road maps. In addition, the government, technologically, has to support R&D for integration with other high technology, standardization of distribution industry's blockchain technology and manpower training to expand technology development.