• 제목/요약/키워드: Data placement

검색결과 611건 처리시간 0.032초

데이터 집약 과학 워크플로우 응용의 특성을 고려한 데이터 배치 기법 (A Data Placement Scheme for the Characteristics of Data Intensive Scientific Workflow Applications)

  • 안주림;김윤희
    • KNOM Review
    • /
    • 제21권2호
    • /
    • pp.46-52
    • /
    • 2018
  • 클라우드 컴퓨팅 환경을 활용한 데이터 집약적인 과학 워크플로우 응용 실험의 경우 클라우드의 여러 데이터 센터에 대량의 데이터가 분산될 수 있고, 생성되는 중간 데이터는 서로 다른 데이터 센터 간의 접근을 통해 전송될 수 있다. 또한 응용의 실행이 진행될 때, 생성된 중간 데이터를 이용하며 진행되므로 데이터의 위치에 따라 실행 결과가 달라진다. 그러나 기존의 데이터 배치 기법은 과학 응용의 특성을 고려하지 않는다. 본 논문에서는 데이터 집약적 단계를 정의하여 그 구간에서의 런타임 데이터 배치를 제안한다. 제안하는 데이터 배치 기법을 통해 본 연구에서 정의한 데이터 집약적 단계에서의 횟수를 고려한 시나리오를 분석하여 결과를 도출한다. 또한 런타임 데이터 배치 횟수와 런타임 데이터 배치 시 오버헤드를 분석하여 성능을 비교했다.

Virtual Machine Placement Methods using Metaheuristic Algorithms in a Cloud Environment - A Comprehensive Review

  • Alsadie, Deafallah
    • International Journal of Computer Science & Network Security
    • /
    • 제22권4호
    • /
    • pp.147-158
    • /
    • 2022
  • Cloud Computing offers flexible, on demand, ubiquitous resources for cloud users. Cloud users are provided computing resources in a virtualized environment. In order to meet the growing demands for computing resources, data centres contain a large number of physical machines accommodating multiple virtual machines. However, cloud data centres cannot utilize their computing resources to their total capacity. Several policies have been proposed for improving energy proficiency and computing resource utilization in cloud data centres. Virtual machine placement is an effective method involving efficient mapping of virtual machines to physical machines. However, the availability of many physical machines accommodating multiple virtual machines in a data centre has made the virtual machine placement problem a non deterministic polynomial time hard (NP hard) problem. Metaheuristic algorithms have been widely used to solve the NP hard problems of multiple and conflicting objectives, such as the virtual machine placement problem. In this context, we presented essential concepts regarding virtual machine placement and objective functions for optimizing different parameters. This paper provides a taxonomy of metaheuristic algorithms for the virtual machine placement method. It is followed by a review of prominent research of virtual machine placement methods using meta heuristic algorithms and comparing them. Finally, this paper provides a conclusion and future research directions in virtual machine placement of cloud computing.

A cache placement algorithm based on comprehensive utility in big data multi-access edge computing

  • Liu, Yanpei;Huang, Wei;Han, Li;Wang, Liping
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제15권11호
    • /
    • pp.3892-3912
    • /
    • 2021
  • The recent rapid growth of mobile network traffic places multi-access edge computing in an important position to reduce network load and improve network capacity and service quality. Contrasting with traditional mobile cloud computing, multi-access edge computing includes a base station cooperative cache layer and user cooperative cache layer. Selecting the most appropriate cache content according to actual needs and determining the most appropriate location to optimize the cache performance have emerged as serious issues in multi-access edge computing that must be solved urgently. For this reason, a cache placement algorithm based on comprehensive utility in big data multi-access edge computing (CPBCU) is proposed in this work. Firstly, the cache value generated by cache placement is calculated using the cache capacity, data popularity, and node replacement rate. Secondly, the cache placement problem is then modeled according to the cache value, data object acquisition, and replacement cost. The cache placement model is then transformed into a combinatorial optimization problem and the cache objects are placed on the appropriate data nodes using tabu search algorithm. Finally, to verify the feasibility and effectiveness of the algorithm, a multi-access edge computing experimental environment is built. Experimental results show that CPBCU provides a significant improvement in cache service rate, data response time, and replacement number compared with other cache placement algorithms.

데이타 종류에 기반한 NOD 시스템의 데이타 배치 방법 (A Data Placement Method of NOD systems based on data types)

  • 장시웅
    • 한국정보통신학회논문지
    • /
    • 제3권2호
    • /
    • pp.421-431
    • /
    • 1999
  • NOD(News On Demand) 시스템은 텍스트, 이미지, 그리고 비디오 등의 다양한 형태의 데이타를 포함하며, 데이타의 형태에 따라 크기가 다르다. 따라서, 본 논문에서는 데이타의 종류에 따라 데이타의 배치 방법을 다르게 하는 DPMDT(Data Placement Method based on Data Types)방법을 제안하고, 생성된 날짜를 기반으로 데이타를 배치하는 TBSM(Time Based Storage Management) 방법과 생성날짜와 접속 비율을 기반으로 데이타를 배치하는 RBSM(Rate Based Storage Management) 방법과의 성능을 비교, 분석하였다. 분석 결과, 비디오 뉴스의 재생 시간이 길고 디스크의 개수가 적은 경우(1개)에는 DPMDT 방법이 TBSM이나 RBSM 방법에 비해 다소 낮은 성능을 보였으나, 디스크의 개수가 2개 이상인 경우에는 DPMDT 방법이 TBSM이나 RBSM 방법에 비해 좋은 성능을 보였다.

  • PDF

RDP: A storage-tier-aware Robust Data Placement strategy for Hadoop in a Cloud-based Heterogeneous Environment

  • Muhammad Faseeh Qureshi, Nawab;Shin, Dong Ryeol
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제10권9호
    • /
    • pp.4063-4086
    • /
    • 2016
  • Cloud computing is a robust technology, which facilitate to resolve many parallel distributed computing issues in the modern Big Data environment. Hadoop is an ecosystem, which process large data-sets in distributed computing environment. The HDFS is a filesystem of Hadoop, which process data blocks to the cluster nodes. The data block placement has become a bottleneck to overall performance in a Hadoop cluster. The current placement policy assumes that, all Datanodes have equal computing capacity to process data blocks. This computing capacity includes availability of same storage media and same processing performances of a node. As a result, Hadoop cluster performance gets effected with unbalanced workloads, inefficient storage-tier, network traffic congestion and HDFS integrity issues. This paper proposes a storage-tier-aware Robust Data Placement (RDP) scheme, which systematically resolves unbalanced workloads, reduces network congestion to an optimal state, utilizes storage-tier in a useful manner and minimizes the HDFS integrity issues. The experimental results show that the proposed approach reduced unbalanced workload issue to 72%. Moreover, the presented approach resolve storage-tier compatibility problem to 81% by predicting storage for block jobs and improved overall data block placement by 78% through pre-calculated computing capacity allocations and execution of map files over respective Namenode and Datanodes.

Information entropy based algorithm of sensor placement optimization for structural damage detection

  • Ye, S.Q.;Ni, Y.Q.
    • Smart Structures and Systems
    • /
    • 제10권4_5호
    • /
    • pp.443-458
    • /
    • 2012
  • The structural health monitoring (SHM) benchmark study on optimal sensor placement problem for the instrumented Canton Tower has been launched. It follows the success of the modal identification and model updating for the Canton Tower in the previous benchmark study, and focuses on the optimal placement of vibration sensors (accelerometers) in the interest of bettering the SHM system. In this paper, the sensor placement problem for the Canton Tower and the benchmark model for this study are first detailed. Then an information entropy based sensor placement method with the purpose of damage detection is proposed and applied to the benchmark problem. The procedure that will be implemented for structural damage detection using the data obtained from the optimal sensor placement strategy is introduced and the information on structural damage is specified. The information entropy based method is applied to measure the uncertainties throughout the damage detection process with the use of the obtained data. Accordingly, a multi-objective optimal problem in terms of sensor placement is formulated. The optimal solution is determined as the one that provides equally most informative data for all objectives, and thus the data obtained is most informative for structural damage detection. To validate the effectiveness of the optimally determined sensor placement, damage detection is performed on different damage scenarios of the benchmark model using the noise-free and noise-corrupted measured information, respectively. The results show that in comparison with the existing in-service sensor deployment on the structure, the optimally determined one is capable of further enhancing the capability of damage detection.

보건교사 배치기준의 변천과정 및 배치정책의 문제 연구 (A Study on the Change in Health Teacher Placement Standards and the Problems in the Placement Policy)

  • 김미경
    • 한국학교보건학회지
    • /
    • 제26권3호
    • /
    • pp.133-143
    • /
    • 2013
  • Purpose: The purpose of this study is to provide basic data for a more reasonable health teacher placement policy sending teachers to more appropriate sites, by analyzing the change process of the health teacher placement standards and the problems caused by an unreasonable placement policy. Methods: This study mainly analyzed relevant research data and existing studies focusing on a literature analysis. Results: To date, the placement policy for health teachers has changed, going through expansion, reduction, and retrogression, since its establishment. The standard, placing health teachers only in elementary schools with more than 18 classes, was created in 1952. Despite the expansion of the role of health teachers and the revision of the school health law in 2007, this standard has been applied to date without modification. In the meantime, there have been many problems caused by inappropriate placement of health teachers. It was difficult for health teachers in large schools to carry out proper health education; and, in many schools, passive health management, such as first aid, health tests, and student health management, was mainly executed rather than active health management. Students in small schools were not even given an opportunity to receive health education and health management owing to the absence of health teachers. Also, compared to teachers teaching other subjects, health teachers have had very unfair placement standards. Conclusion: The placement policy for health teachers, which has been applied to the present, has never reflected social change, the increase of student health issues, and the demand from the school area. Although the role of health teachers expanded with the execution of health education, the current placement standards for health teachers are very unreasonable. Accordingly, it is necessary to review the health teacher placement policy in a reasonable manner and to revise the standards considering the reality.

  • PDF

Scalable Service Placement in the Fog Computing Environment for the IoT-Based Smart City

  • Choi, Jonghwa;Ahn, Sanghyun
    • Journal of Information Processing Systems
    • /
    • 제15권2호
    • /
    • pp.440-448
    • /
    • 2019
  • The Internet of Things (IoT) is one of the main enablers for situation awareness needed in accomplishing smart cities. IoT devices, especially for monitoring purposes, have stringent timing requirements which may not be met by cloud computing. This deficiency of cloud computing can be overcome by fog computing for which fog nodes are placed close to IoT devices. Because of low capabilities of fog nodes compared to cloud data centers, fog nodes may not be deployed with all the services required by IoT devices. Thus, in this article, we focus on the issue of fog service placement and present the recent research trends in this issue. Most of the literature on fog service placement deals with determining an appropriate fog node satisfying the various requirements like delay from the perspective of one or more service requests. In this article, we aim to effectively place fog services in accordance with the pre-obtained service demands, which may have been collected during the prior time interval, instead of on-demand service placement for one or more service requests. The concept of the logical fog network is newly presented for the sake of the scalability of fog service placement in a large-scale smart city. The logical fog network is formed in a tree topology rooted at the cloud data center. Based on the logical fog network, a service placement approach is proposed so that services can be placed on fog nodes in a resource-effective way.

A Network Load Sensitive Block Placement Strategy of HDFS

  • Meng, Lingjun;Zhao, Wentao;Zhao, Haohao;Ding, Yang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제9권9호
    • /
    • pp.3539-3558
    • /
    • 2015
  • This paper investigates and analyzes the default block placement strategy of HDFS. HDFS is a typical representative distributed file system to stream vast amount of data effectively at high bandwidth to user applications. However, the default HDFS block placement policy assumes that all nodes in the cluster are homogeneous, and places blocks with a simple RoundRobin strategy without considering any nodes' resource characteristics, which decreases self-adaptability of the system. The primary contribution of this paper is the proposition of a network load sensitive block placement strategy. We have implemented our algorithm and justify it through extensive simulations and comparison with similar existing studies. The results indicate that our work not only performs much better in the data distribution but also improves write performance more significantly than the others.

Performance Optimization of Big Data Center Processing System - Big Data Analysis Algorithm Based on Location Awareness

  • Zhao, Wen-Xuan;Min, Byung-Won
    • International Journal of Contents
    • /
    • 제17권3호
    • /
    • pp.74-83
    • /
    • 2021
  • A location-aware algorithm is proposed in this study to optimize the system performance of distributed systems for processing big data with low data reliability and application performance. Compared with previous algorithms, the location-aware data block placement algorithm uses data block placement and node data recovery strategies to improve data application performance and reliability. Simulation and actual cluster tests showed that the location-aware placement algorithm proposed in this study could greatly improve data reliability and shorten the application processing time of I/O interfaces in real-time.