• Title/Summary/Keyword: distributed file system

Search Result 251, Processing Time 0.04 seconds

Fair Peer Assignment Scheme for Peer-to-Peer File Sharing

  • Hu, Chih-Lin;Chen, Da-You;Chang, Yi-Hsun;Chen, Yu-Wen
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.4 no.5
    • /
    • pp.709-735
    • /
    • 2010
  • The reciprocal virtue of peer-to-peer networking has stimulated an explosion of peer population and service capacity, ensuring rapid content distribution in peer-to-peer networks. Critical issues such as peer churn, free riding, and skewed workload significantly affect performance results such as service agility, fairness, and resource utilization. To resolve these problems systematically, this study proposes a peer assignment scheme that supports fair peer-to-peer file sharing applications. The proposed scheme exploits the peer duality of both server-oriented peer capacity and client-oriented peer contribution. Accordingly, the system server can prioritize download requests and appropriately assign server peers to uploading file objects. Several functional extensions, including peer substitution and elimination, bandwidth adjustment, and distributed modification, help cope with subtle situations of service starvation and download blocking, and hence make the system design robust and amenable. Simulation results show this design is examined under both centralized and distributed peer-to-peer environments. Performance results confirm that the proposed mechanisms are simple but effective in maintaining service agility and fairness, without loss of overall service capacity in peer-to-peer files sharing systems.

A Design and its Implementation of Java based Management System for Distributed File Systems (Distributed file System을 위한 Java 기반 Management System설계 및 구현)

  • 김태형;정규식
    • Proceedings of the IEEK Conference
    • /
    • 2002.06c
    • /
    • pp.149-152
    • /
    • 2002
  • 최근 인터넷 사용자가 늘어나면서 클러스터링 기반의 대형 서버가 등장하게 되었고 또한 분산파일시스템(DFS)에 대한 관심이 커지게 되었다. 기존 분산파일 시스템들은 파일시스템 측면의 많은 기능들이 제공되고 뛰어난 성능을 발휘하지만 사용자 인터페이스 및 관리 측면에서 본다면 미흡한 점이 많다. 본 논문에서는 분산파일 시스템의 종류와 플랫폼에 상관없이, 사용하기 쉬운 DFS Management System을 설계하고 구축한다. 전체 시스템 구조를 파악 할 수 있도록 GUI 환경을 제공하며, 모듈화/계층화 구조로 설계하는데 기존 DFS와 연동하기 위한 하부 통신모듈이 포함된다. 본 논문에서는 Java기반으로 DFS Management System을 구축하였으며, Coda DFS가 설치된 클러스터링 서버와 연동하여 시험하였다.

  • PDF

Distributed Identity Authentication System based on DID Technology (DID 기술에 기반 한 분산 신원 인증 시스템)

  • Chai Ting;Seung-Soon Shin;Sung-Hwa Han
    • Convergence Security Journal
    • /
    • v.23 no.4
    • /
    • pp.17-22
    • /
    • 2023
  • Traditional authentication systems typically involve users entering their username and password into a centralized identity management system. To address the inconvenience of such authentication methods, a decentralized identity authentication system based on Distributed Identifiers(DID) is proposed, utilizing decentralized identity technology. The proposed system employs QR code scanning for login, enhancing security through the use of blockchain technology to ensure the uniqueness and safety of user identities during the login process. This system utilizes DIDs and integrates the InterPlanetary File System(IPFS) to securely manage organizational members' identity information while keeping it private. Using the distributed identity authentication system proposed in this study, it is possible to effectively manage the security and personal identity of organization members. To improve the usability of the system proposed in this study, research is needed to expand it into a solution.

Delayed Block Replication Scheme of Hadoop Distributed File System for Flexible Management of Distributed Nodes (하둡 분산 파일시스템에서의 유연한 노드 관리를 위한 지연된 블록 복제 기법)

  • Ryu, Woo-Seok
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.12 no.2
    • /
    • pp.367-374
    • /
    • 2017
  • This paper discusses management problems of Hadoop distributed node, which is a platform for big data processing, and proposes a novel technique for enabling flexible node management of Hadoop Distributed File System. Hadoop cannot configure Hadoop cluster dynamically because it judges temporarily unavailable nodes as a failure. Delayed block replication scheme proposed in this paper delays the removal of unavailable node as much as possible so as to be easily rejoined. Experimental results show that the proposed scheme increases flexibility of node management with little impact on distributed processing performance when the cluster size changes.

A Study on File Allocation Algorithm in Distributed Computer Systems (분산 컴퓨터 시스템에 있어서의 화일 할당 알고리듬에 관한 연구)

  • Hong, Jin-Pyo;Lim, Chae-Tak
    • Journal of the Korean Institute of Telematics and Electronics
    • /
    • v.27 no.2
    • /
    • pp.118-125
    • /
    • 1990
  • An optimal file allocation algorithm which seeks optimal solution of file allocation problem for efficient management and operation of information files in distributed computer system is proposed. Since file allocation time in practical applications that have many computer sites is tool long, the problem size has to be reduced and computation time is improved by using preassignment conditio. A new method which calculate appriasal value for accurrate value for accurrate representation of assigned state is proposed and the selection criteria to candidate nodes for rapid determination of allocation are given. By using selection criteria, file allocation is determined and final appraisal value represent total cost of assigned state.

  • PDF

A Distributed Cache Management Scheme for Efficient Accesses of Small Files in HDFS (HDFS에서 소형 파일의 효율적인 접근을 위한 분산 캐시 관리 기법)

  • Oh, Hyunkyo;Kim, Kiyeon;Hwang, Jae-Min;Park, Junho;Lim, Jongtae;Bok, Kyoungsoo;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.14 no.11
    • /
    • pp.28-38
    • /
    • 2014
  • In this paper, we propose the distributed cache management scheme to efficiently access small files in Hadoop Distributed File Systems(HDFS). The proposed scheme can reduce the number of metadata managed by a name node since many small files are merged and stored in a chunk. It is also possible to reduce the file access costs, by keeping the information of requested files using the client cache and data node caches. The client cache keeps small files that a user requests and metadata. Each data node cache keeps the small files that are frequently requested by users. It is shown through performance evaluation that the proposed scheme significantly reduces the processing time over the existing scheme.

Design and implementation of a Shared-Concurrent File System in distributed UNIX environment (분산 UNIX 환경에서 Shared-Concurrent File System의 설계 및 구현)

  • Jang, Si-Ung;Jeong, Gi-Dong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.3 no.3
    • /
    • pp.617-630
    • /
    • 1996
  • In this paper, a shared-concurrent file system (S-CFS) is designed and implemented using conventional disks as disk arrays on a Workstation Cluster which can be used as a small-scale server. Since it is implemented on UNIX operating systems, S_CFS is not only portable and flexible but also efficient in resource usage because it does not require additional I/O nodes. The result of the research shows that on small-scale systems with enough disks, the performance of the concurrent file system on transaction processing applications is bounded by the bottleneck of CPUs computing powers while the performance of the concurrent file system on massive data I/Os is bounded by the time required to copy data between buffers. The concurrent file system,which has been implemented on a Workstation Cluster with 8 disks,shows a throughput of 388 tps in case of transaction processing applications and can provide the bandwidth of 15.8 Mbytes/sec in case of massive data processing applications. Moreover,the concurrent file system has been dsigned to enhance the throughput of applications requirring high performance I/O by controlling the paralleism of the concurrent file system on user's side.

  • PDF

Design of Distributed Cloud System for Managing large-scale Genomic Data

  • Seine Jang;Seok-Jae Moon
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.16 no.2
    • /
    • pp.119-126
    • /
    • 2024
  • The volume of genomic data is constantly increasing in various modern industries and research fields. This growth presents new challenges and opportunities in terms of the quantity and diversity of genetic data. In this paper, we propose a distributed cloud system for integrating and managing large-scale gene databases. By introducing a distributed data storage and processing system based on the Hadoop Distributed File System (HDFS), various formats and sizes of genomic data can be efficiently integrated. Furthermore, by leveraging Spark on YARN, efficient management of distributed cloud computing tasks and optimal resource allocation are achieved. This establishes a foundation for the rapid processing and analysis of large-scale genomic data. Additionally, by utilizing BigQuery ML, machine learning models are developed to support genetic search and prediction, enabling researchers to more effectively utilize data. It is expected that this will contribute to driving innovative advancements in genetic research and applications.

CORBA 및 DCOM 기반 제어 시스템

  • 박홍성;강원준;김형육;김경식
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2000.10a
    • /
    • pp.546-546
    • /
    • 2000
  • This paper designs and implements special objects such as a control object, a I/O object and a file object based on CORBA and DCOM which are suitable to distributed control systems. And interactions among those objects ate analyzed.

  • PDF

분산 데이타베이스에서의 동적 화일배정에 관한 연구

  • 황영헌;김대환;김영호;강석호
    • Proceedings of the Korean Operations and Management Science Society Conference
    • /
    • 1996.10a
    • /
    • pp.275-278
    • /
    • 1996
  • We propose dynamic file allocation method in distributed database management system with changing access patterns. There are a lot of studies on file allocation problem in D-DBMS, and those studies deal with off-line analysis and optimization. Those works are well for systems with static database access patterns, but are inadequate for systems that have changing access patterns. In these systems, dynamic file allocation along with access pattern is more proper. In advance, Brunstrom et al. studied on this area, but they dealt a extremely simplified model. So, we make more practical models to simulate real system. In these models, many factors that were disregard in the advance study are considered. These models are composed with the non-replication system and the replication system. In addition to, we deal with CPU workload balancing in such system in order to improve performance of systems. Our methodology is very simple and realistic, therefore we think that it will give a lot of improvement in D-DBMS with changing access pattern.

  • PDF