• Title/Summary/Keyword: Distributed File Systems

Search Result 123, Processing Time 0.024 seconds

Energy and Performance-Efficient Dynamic Load Distribution for Mobile Heterogeneous Storage Devices (에너지 및 성능 효율적인 이종 모바일 저장 장치용 동적 부하 분산)

  • Kim, Young-Jin;Kim, Ji-Hong
    • Journal of the Korea Society of Computer and Information
    • /
    • v.14 no.4
    • /
    • pp.9-17
    • /
    • 2009
  • In this paper, we propose a dynamic load distribution technique at the operating system level in mobile storage systems with a heterogeneous storage pair of a small form-factor and disk and a flash memory, which aims at saving energy consumption as well as enhancing I/O performance. Our proposed technique takes a combinatory approach of file placement and buffer cache management techniques to find how the load can be distributed in an energy and performance-aware way for a heterogeneous mobile storage air of a hard disk and a flash memory. We demonstrate that the proposed technique provides better experimental results with heterogeneous mobile storage devices compared with the existing techniques through extensive simulations.

An Efficient Cache Management Scheme for Load Balancing in Distributed Environments with Different Memory Sizes (상이한 메모리 크기를 가지는 분산 환경에서 부하 분산을 위한 캐시 관리 기법)

  • Choi, Kitae;Yoon, Sangwon;Park, Jaeyeol;Lim, Jongtae;Lee, Seokhee;Bok, Kyoungsoo;Yoo, Jaesoo
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.8
    • /
    • pp.543-548
    • /
    • 2015
  • Recently, volume of data has been growing dramatically along with the growth of social media and digital devices. However, the existing disk-based distributed file systems have limits to their performance of data processing or data access, due to I/O processing costs and bottlenecks. To solve this problem, the caching technique is being used to manage data in the memory. In this paper, we propose a cache management scheme to handle load balancing in a distributed memory environment. The proposed scheme distributes the data according to the memory size, n distributed environments with different memory sizes. If overloaded nodes occur, it redistributes the the access time of the caching data. In order to show the superiority of the proposed scheme, we compare it with an existing distributed cache management scheme through performance evaluation.

Extending UML Interaction Diagrams For Mobile Agents Including Agent Platforms (플랫폼을 포함한 이동 에이전트를 위한 UML 상호작용 다이어그램의 확장)

  • Yoo, Moon-Sung
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.11 no.2
    • /
    • pp.261-267
    • /
    • 2011
  • One of powerful software paradigms for distributed systems is a mobile agent system. Since the usage of mobile agent systems is increased, a software development model to construct softwares efficiently for these systems is needed. Currently, UML is a widely used software development model. However, existing UML can not describe the necessary mobility of the mobile agent based software systems in explicit way. In this paper, the interaction diagrams of UML(sequence diagrams and communication diagrams) are extended and used to express the mobility of the mobile agents including the functions of platforms of mobile agent systems in three ways. For a case study, we applied the extended diagrams to a distributed file searching using mobile agents, and we confirmed these diagrams can describe the function and mobility of mobile agents very well.

Development of Distributed Rainfall-Runoff Modelling System Integrated with GIS (지리정보시스템과 통합된 분포형 강우-유출 모의 시스템 개발)

  • Choi, Yun-Seok;Kim, Kyung-Tak;Shim, Myung-Pil
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.12 no.3
    • /
    • pp.76-87
    • /
    • 2009
  • Most distributed models have been developed for data interchange between model for hydrological analysis and GIS(Geographic Information System). And some interface systems between them have been developed to operate the model conveniently. This study is about developing integrated system between model and GIS not coupled system based on file interchange or interface system. In this study, HyGIS-GRM which is integrated system between GRM(Grid based Rainfall-runoff Model) which is physically based distributed rainfall-runoff model and HyGIS(Hydro Geographic Information System) have been developed. HyGIS-GRM can carry out all the processes from preparing input data to appling them to model in the same system, and this operation environment can improve the efficiency of running the model and analyzing modeling results. HyGIS-GRM can provide objective modeling environment through establishing the process of integrated operation of GIS and distributed model, and we can obtain fundamental technologies for developing integrated system between GIS and water resources model.

  • PDF

Incremental Maintenance of Horizontal Views Using a PIVOT Operation and a Differential File in Relational DBMSs (관계형 데이터베이스에서 PIVOT 연산과 차등 파일을 이용한 수평 뷰의 점진적인 관리)

  • Shin, Sung-Hyun;Kim, Jin-Ho;Moon, Yang-Sae;Kim, Sang-Wook
    • The KIPS Transactions:PartD
    • /
    • v.16D no.4
    • /
    • pp.463-474
    • /
    • 2009
  • To analyze multidimensional data conveniently and efficiently, OLAP (On-Line Analytical Processing) systems or e-business are widely using views in a horizontal form to represent measurement values over multiple dimensions. These views can be stored as materialized views derived from several sources in order to support accesses to the integrated data. The horizontal views can provide effective accesses to complex queries of OLAP or e-business. However, we have a problem of occurring maintenance of the horizontal views since data sources are distributed over remote sites. We need a method that propagates the changes of source tables to the corresponding horizontal views. In this paper, we address incremental maintenance of horizontal views that makes it possible to reflect the changes of source tables efficiently. We first propose an overall framework that processes queries over horizontal views transformed from source tables in a vertical form. Under the proposed framework, we propagate the change of vertical tables to the corresponding horizontal views. In order to execute this view maintenance process efficiently, we keep every change of vertical tables in a differential file and then modify the horizontal views with the differential file. Because the differential file is represented as a vertical form, its tuples should be converted to those in a horizontal form to apply them to the out-of-date horizontal view. With this mechanism, horizontal views can be efficiently refreshed with the changes in a differential file without accessing source tables. Experimental results show that the proposed method improves average performance by 1.2$\sim$5.0 times over the existing methods.

Design of agent intrusion detection system applying data mining (데이터 마이닝을 적용한 에이전트 침입 탐지 시스템 설계)

  • Jeong Jong Kun;Lee Sung Tae;Kim Yong Ho;Lee Yun Bae
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2001.05a
    • /
    • pp.676-679
    • /
    • 2001
  • As network security is coning up with significant problem after the major Internet sites were hacked nowadays, IDS(Intrusion Detection System) is considered as a next generation security solution for more reliable network and system security rather than firewall. In this paper, we propose the new IDS model which tan detect intrusion in different systems as well as which ran make real-time detection of intrusion in the expanded distributed environment in host level of drawback of existing IDS. We implement its prototype and verify its validity. We use pattern extraction agent so that we can extract automatically audit file needed in distributed intrusion detection even in other platforms.

  • PDF

Development of Big-data Management Platform Considering Docker Based Real Time Data Connecting and Processing Environments (도커 기반의 실시간 데이터 연계 및 처리 환경을 고려한 빅데이터 관리 플랫폼 개발)

  • Kim, Dong Gil;Park, Yong-Soon;Chung, Tae-Yun
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.16 no.4
    • /
    • pp.153-161
    • /
    • 2021
  • Real-time access is required to handle continuous and unstructured data and should be flexible in management under dynamic state. Platform can be built to allow data collection, storage, and processing from local-server or multi-server. Although the former centralize method is easy to control, it creates an overload problem because it proceeds all the processing in one unit, and the latter distributed method performs parallel processing, so it is fast to respond and can easily scale system capacity, but the design is complex. This paper provides data collection and processing on one platform to derive significant insights from various data held by an enterprise or agency in the latter manner, which is intuitively available on dashboards and utilizes Spark to improve distributed processing performance. All service utilize dockers to distribute and management. The data used in this study was 100% collected from Kafka, showing that when the file size is 4.4 gigabytes, the data processing speed in spark cluster mode is 2 minute 15 seconds, about 3 minutes 19 seconds faster than the local mode.

Efficient Load Balancing Scheme using Resource Information in Web Server System (웹 서버 시스템에서의 자원 정보를 이용한 효율적인 부하분산 기법)

  • Chang Tae-Mu;Myung Won-Shig;Han Jun-Tak
    • The KIPS Transactions:PartA
    • /
    • v.12A no.2 s.92
    • /
    • pp.151-160
    • /
    • 2005
  • The exponential growth of Web users requires the web serves with high expandability and reliability. It leads to the excessive transmission traffic and system overload problems. To solve these problems, cluster systems are widely studied. In conventional cluster systems, when the request size is large owing to such types as multimedia and CGI, the particular server load and response time tend to increase even if the overall loads are distributed evenly. In this paper, a cluster system is proposed where each Web server in the system has different contents and loads are distributed efficiently using the Web server resource information such as CPU, memory and disk utilization. Web servers having different contents are mutually connected and managed with a network file system to maintain information consistency required to support resource information updates, deletions, and additions. Load unbalance among contents group owing to distribution of contents can be alleviated by reassignment of Web servers. Using a simulation method, we showed that our method shows up to $50\%$ about average throughput and processing time improvement comparing to systems using each LC method and RR method.

Analyses of Replica Placement Schemes in Distributed File Systems (대용량 분산 파일 시스템의 복제 배치 기법 분석)

  • Choi, Dae-Soon;Kim, Jee-Hong;Eom, Young-Ik
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2012.06a
    • /
    • pp.373-375
    • /
    • 2012
  • 정보의 양이 늘어나고 그 크기가 커짐에 따라 많은 분야에서 대규모 분산 파일 시스템을 제안하고 사용하고 있다. 현재 제안된 분산 파일 시스템은 대부분 확장가능하며 장애에 견딜 수 있고 신뢰성이 있는 시스템을 기본으로 하여 구성되었다. 따라서 분산 파일 시스템은 장애에 효과적으로 견질 수 있고 높은 신뢰성을 보장하기 위해서 데이터의 복제를 만들어 서로 다른 스토리지에 배치한다. 본 논문에서는 대규모 분산 파일 시스템의 복제 배치 기법에 관하여 비교 분석 하였다. 먼저 분산 파일 시스템의 동작 구조를 확인하고 분산 파일 시스템에서 보편적으로 중요시 되는 특성을 파악하였다. 그리고 복제 배치 기법이 효과적인 분산 파일 시스템 구성 시 고려해야 할 것들을 부하 분산, 디스크 분산, 장애 분산으로 나누었다. 이 세 가지 요소를 기준으로 하여 분산 파일 시스템인 GFS, HDFS, Ceph, MooseFS의 복제 배치 기법을 분석하고 각각의 특성 파악하였다.