• Title/Summary/Keyword: Distributed File Systems

Search Result 123, Processing Time 0.026 seconds

A Repository for Workflow Management on Distributed Object Environment (분산객체 환경에서의 워크플로우 관리를 위한 정보저장소)

  • Yeom, Tae-Jin;Park, Jae-Hyung;Ri, Ja;Kim, Ki-Bong;Jin, Sung-Il
    • The Journal of Society for e-Business Studies
    • /
    • v.4 no.1
    • /
    • pp.1-19
    • /
    • 1999
  • Workflow management system provides automation of job processes by maintaining shareability on information about various job process schedules and persons related to those schedules. Existing workflow management systems use file or database to store the information generated in those systems. However, file or database system could manage only non-complicated information for the workflow but not the information resources of an enterprise which is complicated and of various formats. Therefore, we need a data management system that could control those information resources. This system should manage the data which are distributed at several places geographically. Information Repository could meet those requirements. Information Repository may integrate, store and manage information resources requested by application systems. We have an international standard for the information repository, Information Resources Dictionary System(IRDS). The IRDS, however, does not support distributed environment. In this paper, we design and implement an information repository based on IRDS that may be operated in distributed environment. We verify that this information repository is more effective and is more effective than any other file or database system.

  • PDF

Performance Analysis of Distributed Hadoop Systems (분산 하둡 시스템의 성능 비교 분석)

  • Bae, Byoung-Jin;Kim, Young-Joo;Kim, Young-Kuk
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2014.05a
    • /
    • pp.479-482
    • /
    • 2014
  • Nowadays open-source hadoop systems have been using widely to efficiently manage a fast-growing big data. Hadoop systems consist of distributed file processing system called HDFS (Hadoop Distributed File System) and distributed parallel processing system called MapReduce. The MapReduce reads and processes big data from HDFS and then processed results are written in HDFS again by the MapReduce. Such a processing method has different system structure respectively according to hadoop version. Therefore, this paper shows analysis results for performance of hadoop systems. For this, we devise a way which monitors hadoop systems and measure occurrence frequency of processes, threads, and variables generated in hadoop system itself using the devised way. So, by using the measured results as analysis indicator, we help the indicator predict inner performance of hadoop systems.

  • PDF

Distributed File Systems Architectures of the Large Data for Cloud Data Services (클라우드 데이터 서비스를 위한 대용량 데이터 처리 분산 파일 아키텍처 설계)

  • Lee, Byoung-Yup;Park, Jun-Ho;Yoo, Jae-Soo
    • The Journal of the Korea Contents Association
    • /
    • v.12 no.2
    • /
    • pp.30-39
    • /
    • 2012
  • In these day, some of IT venders already were going to cloud computing market, as well they are going to expand their territory for the cloud computing market through that based on their hardware and software technology, making collaboration between hardware and software vender. Distributed file system is very mainly technology for the cloud computing that must be protect performance and safety for high levels service requests as well data store. This paper introduced distributed file system for cloud computing and how to use this theory such as memory database, Hadoop file system, high availability database system. now In the market, this paper define a very large distributed processing architect as a reference by kind of distributed file systems through using technology in cloud computing market.

A Scheme on High-Performance Caching and High-Capacity File Transmission for Cloud Storage Optimization (클라우드 스토리지 최적화를 위한 고속 캐싱 및 대용량 파일 전송 기법)

  • Kim, Tae-Hun;Kim, Jung-Han;Eom, Young-Ik
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.37 no.8C
    • /
    • pp.670-679
    • /
    • 2012
  • The recent dissemination of cloud computing makes the amount of data storage to be increased and the cost of storing the data grow rapidly. Accordingly, data and service requests from users also increases the load on the cloud storage. There have been many works that tries to provide low-cost and high-performance schemes on distributed file systems. However, most of them have some weaknesses on performing parallel and random data accesses as well as data accesses of frequent small workloads. Recently, improving the performance of distributed file system based on caching technology is getting much attention. In this paper, we propose a CHPC(Cloud storage High-Performance Caching) framework, providing parallel caching, distributed caching, and proxy caching in distributed file systems. This study compares the proposed framework with existing cloud systems in regard to the reduction of the server's disk I/O, prevention of the server-side bottleneck, deduplication of the page caches in each client, and improvement of overall IOPS. As a results, we show some optimization possibilities on the cloud storage systems based on some evaluations and comparisons with other conventional methods.

Torus Network Based Distributed Storage System for Massive Multimedia Contents (토러스 연결망 기반의 대용량 멀티미디어용 분산 스토리지 시스템)

  • Kim, Cheiyol;Kim, Dongoh;Kim, Hongyeon;Kim, Youngkyun;Seo, Daewha
    • Journal of Korea Multimedia Society
    • /
    • v.19 no.8
    • /
    • pp.1487-1497
    • /
    • 2016
  • Explosively growing service of digital multimedia data increases the need for highly scalable low-cost storage. This paper proposes the new storage architecture based on torus network which does not need network switch and erasure coding for efficient storage usage for high scalability and efficient disk utilization. The proposed model has to compensate for the disadvantage of long network latency and network processing overhead of torus network. The proposed storage model was compared to two most popular distributed file system, GlusterFS and Ceph distributed file systems through a prototype implementation. The performance of prototype system shows outstanding results than erasure coding policy of two file systems and mostly even better results than replication policy of them.

A Study on Efficient File Allocation for Distributed Computer Systems (분산 컴퓨터 시스템에서 효율적 파일 할당에 관한 연구)

  • 홍진표;임재택
    • Journal of the Korean Institute of Telematics and Electronics
    • /
    • v.26 no.9
    • /
    • pp.1395-1401
    • /
    • 1989
  • An efficient file allocation algorithm and a new method which calculate appraisal value of candidate computer site for distributed computer systems are proposed. The file allocation problem size is reduced by using the preassignment condition. The appraisal value of candidate node is calcualted as the user state array and node state array are varied according to control variables. As the selection criteria is applied to the candidates, the reasonable node is selected and assign state is determined. The proposed algorithm is heuriatic polynomial time algorithm. By performing algorithm for sample problems. It is shown that the proposed algorithm is superior to conventional method in terms of deviation from optimal solution.

  • PDF

CORBA 및 DCOM 기반 제어 시스템

  • 박홍성;강원준;김형육;김경식
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2000.10a
    • /
    • pp.546-546
    • /
    • 2000
  • This paper designs and implements special objects such as a control object, a I/O object and a file object based on CORBA and DCOM which are suitable to distributed control systems. And interactions among those objects ate analyzed.

  • PDF

A Real-time Video Playback Scheme in a Distributed Storage System Supporting File Sharing (파일 공유를 지원하는 분산 저장 시스템에서 실시간 비디오 재생 기법)

  • Eunsam Kim
    • Smart Media Journal
    • /
    • v.12 no.11
    • /
    • pp.145-153
    • /
    • 2023
  • In a P2P-based distributed storage system where peers frequently join and leave, it is essential to guarantee not only data availability but also playback quality comparable to that provided by local storage devices when playing back video files with real-time constraints. In addition, cloud storage services based on distributed storage systems provide each user with the functionality to share their files with other users, so when multiple users request playback of the same video file at the same time, all playback should be supported seamlessly in real time. Therefore, in this paper, we propose a scheme that process multiple simultaneous playback requests for each video file in real time as well as data availability in a P2P-based distributed storage system that supports file sharing. This scheme can support real-time simultaneous playback and efficiently use storage space by adjusting the amount of redundant data encoded through erasure coding according to the number of concurrent playback requests for each video file.

A Measurement Study of User Behavior and File Pollution in DHT-based P2P Networks (DHT 기반의 P2P 네트워크에서 사용자 행동양식 및 파일 오염에 관한 측정 연구)

  • Shin, Kyu-Yong;Yoo, Jin-Cheol;Lee, Jong-Deog
    • Journal of the Korea Society of Computer and Information
    • /
    • v.16 no.3
    • /
    • pp.131-140
    • /
    • 2011
  • File pollution (i.e., sharing of corrupted files, or contaminating index information with bogus index records) is a de facto problem in many file sharing Peer-to-Peer (P2P) systems in use today. Since pollution squanders network resources and frustrates users with unprofitable downloads (due to corrupted files) and unproductive download trials (due to bogus index records), the viability of P2P systems (and similar distributed information-sharing applications) is questionable unless properly addressed. Thus, developing effective anti-pollution mechanisms is an immediate problem in this literature. This paper provides useful information and deep insight with future researchers who want to design an effective anti-pollution mechanism throughout an extensive measurement study of user behavior and file pollution in a representative DHT-based P2P system, the Kad network.

A Hoarding Policy for collaborative computing in Mobile Environment : design and Implementation. (이동컴퓨터 상에서의 공동작업을 위한 자동저장 방식 설계 및 구현)

  • 이근영;김남광박승규
    • Proceedings of the IEEK Conference
    • /
    • 1998.10a
    • /
    • pp.491-494
    • /
    • 1998
  • This paper describes the design and implementation of file system which allows the collaborative computing in mobile environment. The design goal is to make a logically one file system in the distributed computer systems. The characteristics of frequent, foreseeable and variable disconnections in a mobile environment wrer taken into consideration. We introduce an auto-hoarding system that provides the availability of large number of nodes which are weakly and intermittently connected. The data consistency problems in distributed or replicated mobile data are also discussed.

  • PDF