• Title/Summary/Keyword: distributed file system

Search Result 251, Processing Time 0.021 seconds

A Measurement Study of User Behavior and File Pollution in DHT-based P2P Networks (DHT 기반의 P2P 네트워크에서 사용자 행동양식 및 파일 오염에 관한 측정 연구)

  • Shin, Kyu-Yong;Yoo, Jin-Cheol;Lee, Jong-Deog
    • Journal of the Korea Society of Computer and Information
    • /
    • v.16 no.3
    • /
    • pp.131-140
    • /
    • 2011
  • File pollution (i.e., sharing of corrupted files, or contaminating index information with bogus index records) is a de facto problem in many file sharing Peer-to-Peer (P2P) systems in use today. Since pollution squanders network resources and frustrates users with unprofitable downloads (due to corrupted files) and unproductive download trials (due to bogus index records), the viability of P2P systems (and similar distributed information-sharing applications) is questionable unless properly addressed. Thus, developing effective anti-pollution mechanisms is an immediate problem in this literature. This paper provides useful information and deep insight with future researchers who want to design an effective anti-pollution mechanism throughout an extensive measurement study of user behavior and file pollution in a representative DHT-based P2P system, the Kad network.

Prototype Design of Mass Distributed Storage System based on PC using Ceph for SMB

  • Cha, ByungRae;Kim, Yongil
    • Smart Media Journal
    • /
    • v.4 no.3
    • /
    • pp.62-67
    • /
    • 2015
  • The trend keywords in ICT sector will be Big Data, Internet of Things, and Cloud Computing. The rear end to support those techniques requires a large-capacity storage technology of low-cost. Therefore, we proposed the prototype of low-cost and mass distributed storage system based on PC using open-source Ceph FS for SMB.

Dynamic File Allocation Problems In Distributed Systems (분산 시스템의 동적 파일 할당 연구)

  • Seo, Pil-Kyo
    • The Transactions of the Korea Information Processing Society
    • /
    • v.4 no.7
    • /
    • pp.1681-1693
    • /
    • 1997
  • In a distributed system, the simple file allocation problem determines the placement of copies of a file, so as to minimize the operating costs. The simple file allocation problem assumes the cost parameters to be fixed. In practice, these parameters change over time. In this research, dynamic file allocation problem for both single and multiple files are considered, which account for these changing parameters. A model for dynamic file allocation problem is formulated as a mixed integer program for which Lagrangian relaxation based branch-and-bound algorithm is developed. This algorithms is implemented and its efficiency is tested on medium to large test problems.

  • PDF

Data Resource Management under Distributed Computing Environment (분산 컴퓨팅 환경하에서의 데이타 자원 관리)

  • 조희경;안중호
    • Proceedings of the Korea Database Society Conference
    • /
    • 1994.09a
    • /
    • pp.105-129
    • /
    • 1994
  • The information system of corporations are facing a new environment expressed by miniaturization, decentralization and Open System. It is therefore of utmost importance for corporations to adapt flexibly th such new environment by providing for corresponding changes to their existing information systems. The objectives of this study are to identify this new environment faced by today′s information system and develop effective methods for data resource management under this new environment. In this study, it is assumed that the new environment faced by information systems can be specified as Distributed Computing Environment, and in order to achieve such system, presents Client/server architecture as its representative computing structure, This study defines Client/server architecture as a computing architecture which specialize the fuctionality of the client system and the server system in order to have an application distribute and perform cooperative processing at the best platform. Furthermore, from among the five structures utilized in Client/server architecture for distribution and cooperative processing of application between server and client this study presents two different data management methods under the Client/server environment; one is "Remote Data Management Method" which uses file server or database server and. the other is "Distributed Data Management Method" using distributed database management system. The result of this study leads to the conclusion that in the client/server environment although distributed application is assumed, the data could become centralized (in the case of file server or database server) or decentralized (in the case of distributed database system) and the data management method through a distributed database system where complete responsibility and powers with respect to control of data used by the user are given not only is it more adaptable to modern flexible corporate environment, but in terms of system operation, it presents a more efficient data management alternative compared to existing data management methods in terms of cutting costs.

  • PDF

A Study on the Improving Performance of Massively Small File Using the Reuse JVM in MapReduce (MapReduce에서 Reuse JVM을 이용한 대규모 스몰파일 처리성능 향상 방법에 관한 연구)

  • Choi, Chul Woong;Kim, Jeong In;Kim, Pan Koo
    • Journal of Korea Multimedia Society
    • /
    • v.18 no.9
    • /
    • pp.1098-1104
    • /
    • 2015
  • With the widespread use of smartphones and IoT (Internet of Things), data are being generated on a large scale, and there is increased for the analysis of such data. Hence, distributed processing systems have gained much attention. Hadoop, which is a distributed processing system, saves the metadata of stored files in name nodes; in this case, the main problems are as follows: the memory becomes insufficient; load occurs because of massive small files; scheduling and file processing time increases because of the increased number of small files. In this paper, we propose a solution to address the increase in processing time because of massive small files, and thus improve the processing performance, using the Reuse JVM method provided by Hadoop. Through environment setting, the Reuse JVM method modifies the JVM produced conventionally for every task, so that multiple tasks are reused sequentially in one JVM. As a final outcome, the Reuse JVM method showed the best processing performance when used together with CombineFileInputFormat.

A study on real-time communication of remote station in the distributed control system (분산 제어 시스템에서 원격 제어국의 실시간 통신에 관한 연구)

  • 김내진;김진태;박인갑
    • Journal of the Korean Institute of Telematics and Electronics A
    • /
    • v.31A no.10
    • /
    • pp.21-30
    • /
    • 1994
  • We discussed the Distributed Control System's design on preface and analyzed time of the real-time communication by using designed system. The DCS proposed in this thesis was implemented to network file system having recovery advantage and shared memory method to access file system of a Remote Station with ease. Also, this system minimized the network delay-time by using the real-time VME147 board. In implemented DCS, the performance analysis of real-time process of a Remote Station was done to get the total time for reak-tune communication from a Remote Station to the Central Station after terminating of process. For the analysis of system performance, we experiented by three steps. Firstly, we measuredthe processing the of LOOP function that real-time CPU convertes to-2,500~10.000 values from the input data of the Analog Interface Card. Secondly, we measured the processing time of the LOGIC function and the LOOP function. Lastly, we measured total processing time for communication from a Remote Station to the Centrol Station.

  • PDF

Trickle Write-Back Scheme for Cache Management in Mobile Computing Environments (?이동 컴퓨팅 환경에서 캐쉬 관리를 위한 TWB 기법)

  • Kim, Moon-Jeong;Eom, Young-Ik
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.27 no.1
    • /
    • pp.89-100
    • /
    • 2000
  • Recently, studies on the mobile computing environments that enable mobile hosts to move while retaining its network connection are in progress. In these mobile computing environments, one of the necessary components is the distributed file system supporting mobile hosts, and there are several issues for the design and implementation of the shared file system. Among these issues, there are problems caused by network traffic on limited bandwidth of wireless media. Also, there are consistency maintenance issues that are caused by update-conflicts on the shared files in the distributed file system. In this paper, we propose TWB(Trickle Write-Back) scheme that utilizes weak connectivity for cache management of mobile clients. This scheme focuses on saving bandwidth, reducing waste of disk space, and reducing risks caused by disconnection. For such goals, this scheme lets mobile clients write back intermediate states periodically or on demand while delaying unnecessary write-backs. Meanwhile, this scheme is based on the existing distributed file system architecture and provides transparency.

  • PDF

A Study on the Implementation of PDM Integration Environment in Heterogeneous Distributed Environment (이기종 분산환경에서 PDM 통합환경 구현에 관한 연구)

  • 김형선
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.21 no.45
    • /
    • pp.33-45
    • /
    • 1998
  • The typical characteristic of PDM(Product Data Management) System seperates the databases to store the meta data and applications. Therefore, meta data contains the information for the location of file, user profiles, relationships between the files, and process. PDM utilizes these information efficiently and does file management, configuration management, and process management. In this view, the integration strategy of PDM is to merge data and process. In the view of architecture, the interface between data and application and the actions of each application execute seamlessly. This architecture is viewed as integrated data and process among enterprises and implemented with client/server technology in distributed process environment that interfaced with open object-oriented technology which is developed with business object in the object-oriented infrastructure. In this paper, we studied the definition, function, and scope of PDM and researched the core technologies to implement the PDM integration environment. We also researched the PDM utilization in distributed enterprise environment and implementation of PDM integration environment with this technical background.

  • PDF

Implementation of Data processing of the High Availability for Software Architecture of the Cloud Computing (클라우드 서비스를 위한 고가용성 대용량 데이터 처리 아키텍쳐)

  • Lee, Byoung-Yup;Park, Junho;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.13 no.2
    • /
    • pp.32-43
    • /
    • 2013
  • These days, there are more and more IT research institutions which foresee cloud services as the predominant IT service in the near future and there, in fact, are actual cloud services provided by some IT leading vendors. Regardless of physical location of the service and environment of the system, cloud service can provide users with storage services, usage of data and software. On the other hand, cloud service has challenges as well. Even though cloud service has its edge in terms of the extent to which the IT resource can be freely utilized regardless of the confinement of hardware, the availability is another problem to be solved. Hence, this paper is dedicated to tackle the aforementioned issues; prerequisites of cloud computing for distributed file system, open source based Hadoop distributed file system, in-memory database technology and high availability database system. Also the author tries to body out the high availability mass distributed data management architecture in cloud service's perspective using currently used distributed file system in cloud computing market.

Design of Lightweight CAD Files with Dimensional Verification Capability for Web-Based Collaboration (웹기반 협업을 위한 치수검증이 가능한 경량캐드파일 설계)

  • Song In-Ho;Chung Sung-Chong
    • Transactions of the Korean Society of Mechanical Engineers A
    • /
    • v.30 no.5 s.248
    • /
    • pp.488-495
    • /
    • 2006
  • The demand for the use of 3D CAD data over the Internet environment has been increased. However, transmission of 3D CAD data has delayed the communication effectiveness because of the CAD data size. Lightweight CAD file design methodology is required for rapid transmission in the distributed environment. In this paper, to derive lightweight CAD files from commercial CAD systems, a file translation system producing a native file is constructed first by using the InterOp and API of the ACIS kernel. Using the B-rep model and mesh data extracted from the native file, the lightweight CAD file with topological information is constructed as a binary file. Since the lightweight CAD file retains topological information, it is applied to the dimensional verification, digital mock-ups and visualization of CAD files. Effectiveness of the proposed lightweight CAD file is confirmed through various case studies.