• Title/Summary/Keyword: Data Placement

Search Result 614, Processing Time 0.021 seconds

A Data Placement Scheme for the Characteristics of Data Intensive Scientific Workflow Applications (데이터 집약 과학 워크플로우 응용의 특성을 고려한 데이터 배치 기법)

  • Ahn, Julim;Kim, Yoonhee
    • KNOM Review
    • /
    • v.21 no.2
    • /
    • pp.46-52
    • /
    • 2018
  • For data-intensive scientific workflow application experiments that leverage the cloud computing environment, large amounts of data can be distributed across multiple data centers in the cloud. The generated intermediate data can also be transmitted through access between different data centers. When the application is executed, the execution result is changed according to the location of the data since the intermediate data generated is used. However, existing data placement strategies do not consider the characteristics of scientific applications. In this paper, we define a data-intensive tasks and propose runtime data placement in that interval. Through the proposed data placement scheme, we analyze the scenarios considering the number of times in the data intensive tasks defined in this study and derive the results. In addition, performance was compared by analyzing runtime data placement times and runtime data placement overhead.

Virtual Machine Placement Methods using Metaheuristic Algorithms in a Cloud Environment - A Comprehensive Review

  • Alsadie, Deafallah
    • International Journal of Computer Science & Network Security
    • /
    • v.22 no.4
    • /
    • pp.147-158
    • /
    • 2022
  • Cloud Computing offers flexible, on demand, ubiquitous resources for cloud users. Cloud users are provided computing resources in a virtualized environment. In order to meet the growing demands for computing resources, data centres contain a large number of physical machines accommodating multiple virtual machines. However, cloud data centres cannot utilize their computing resources to their total capacity. Several policies have been proposed for improving energy proficiency and computing resource utilization in cloud data centres. Virtual machine placement is an effective method involving efficient mapping of virtual machines to physical machines. However, the availability of many physical machines accommodating multiple virtual machines in a data centre has made the virtual machine placement problem a non deterministic polynomial time hard (NP hard) problem. Metaheuristic algorithms have been widely used to solve the NP hard problems of multiple and conflicting objectives, such as the virtual machine placement problem. In this context, we presented essential concepts regarding virtual machine placement and objective functions for optimizing different parameters. This paper provides a taxonomy of metaheuristic algorithms for the virtual machine placement method. It is followed by a review of prominent research of virtual machine placement methods using meta heuristic algorithms and comparing them. Finally, this paper provides a conclusion and future research directions in virtual machine placement of cloud computing.

A cache placement algorithm based on comprehensive utility in big data multi-access edge computing

  • Liu, Yanpei;Huang, Wei;Han, Li;Wang, Liping
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.11
    • /
    • pp.3892-3912
    • /
    • 2021
  • The recent rapid growth of mobile network traffic places multi-access edge computing in an important position to reduce network load and improve network capacity and service quality. Contrasting with traditional mobile cloud computing, multi-access edge computing includes a base station cooperative cache layer and user cooperative cache layer. Selecting the most appropriate cache content according to actual needs and determining the most appropriate location to optimize the cache performance have emerged as serious issues in multi-access edge computing that must be solved urgently. For this reason, a cache placement algorithm based on comprehensive utility in big data multi-access edge computing (CPBCU) is proposed in this work. Firstly, the cache value generated by cache placement is calculated using the cache capacity, data popularity, and node replacement rate. Secondly, the cache placement problem is then modeled according to the cache value, data object acquisition, and replacement cost. The cache placement model is then transformed into a combinatorial optimization problem and the cache objects are placed on the appropriate data nodes using tabu search algorithm. Finally, to verify the feasibility and effectiveness of the algorithm, a multi-access edge computing experimental environment is built. Experimental results show that CPBCU provides a significant improvement in cache service rate, data response time, and replacement number compared with other cache placement algorithms.

A Data Placement Method of NOD systems based on data types (데이타 종류에 기반한 NOD 시스템의 데이타 배치 방법)

  • 장시웅
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.3 no.2
    • /
    • pp.421-431
    • /
    • 1999
  • NOD systems contain the data of multiple types such as text, image and video, and the size of NOD data depend on their data types. Therefore, in this paper, we propose a Data Placement Method based on Data Types(DPMDT), in which the data placement method depends on their type. Then, we analyze the performance of DPMDT with that of a Time Based Storage Management(TBSM) in which the data placement method depends on their created date, and that of Rate Based Storage Management(RBSM) in which the data placement method depends on their created date and accessed rate. In case of long playback of video news and a few disks(one disk), our results show that the performance of DPMDT is less efficient than that of TBSM and RBSM methods, however, in case of over 2 disks, the performance of DPMDT is more efficient than that of TBSM and RBSM methods.

  • PDF

RDP: A storage-tier-aware Robust Data Placement strategy for Hadoop in a Cloud-based Heterogeneous Environment

  • Muhammad Faseeh Qureshi, Nawab;Shin, Dong Ryeol
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.9
    • /
    • pp.4063-4086
    • /
    • 2016
  • Cloud computing is a robust technology, which facilitate to resolve many parallel distributed computing issues in the modern Big Data environment. Hadoop is an ecosystem, which process large data-sets in distributed computing environment. The HDFS is a filesystem of Hadoop, which process data blocks to the cluster nodes. The data block placement has become a bottleneck to overall performance in a Hadoop cluster. The current placement policy assumes that, all Datanodes have equal computing capacity to process data blocks. This computing capacity includes availability of same storage media and same processing performances of a node. As a result, Hadoop cluster performance gets effected with unbalanced workloads, inefficient storage-tier, network traffic congestion and HDFS integrity issues. This paper proposes a storage-tier-aware Robust Data Placement (RDP) scheme, which systematically resolves unbalanced workloads, reduces network congestion to an optimal state, utilizes storage-tier in a useful manner and minimizes the HDFS integrity issues. The experimental results show that the proposed approach reduced unbalanced workload issue to 72%. Moreover, the presented approach resolve storage-tier compatibility problem to 81% by predicting storage for block jobs and improved overall data block placement by 78% through pre-calculated computing capacity allocations and execution of map files over respective Namenode and Datanodes.

Information entropy based algorithm of sensor placement optimization for structural damage detection

  • Ye, S.Q.;Ni, Y.Q.
    • Smart Structures and Systems
    • /
    • v.10 no.4_5
    • /
    • pp.443-458
    • /
    • 2012
  • The structural health monitoring (SHM) benchmark study on optimal sensor placement problem for the instrumented Canton Tower has been launched. It follows the success of the modal identification and model updating for the Canton Tower in the previous benchmark study, and focuses on the optimal placement of vibration sensors (accelerometers) in the interest of bettering the SHM system. In this paper, the sensor placement problem for the Canton Tower and the benchmark model for this study are first detailed. Then an information entropy based sensor placement method with the purpose of damage detection is proposed and applied to the benchmark problem. The procedure that will be implemented for structural damage detection using the data obtained from the optimal sensor placement strategy is introduced and the information on structural damage is specified. The information entropy based method is applied to measure the uncertainties throughout the damage detection process with the use of the obtained data. Accordingly, a multi-objective optimal problem in terms of sensor placement is formulated. The optimal solution is determined as the one that provides equally most informative data for all objectives, and thus the data obtained is most informative for structural damage detection. To validate the effectiveness of the optimally determined sensor placement, damage detection is performed on different damage scenarios of the benchmark model using the noise-free and noise-corrupted measured information, respectively. The results show that in comparison with the existing in-service sensor deployment on the structure, the optimally determined one is capable of further enhancing the capability of damage detection.

A Study on the Change in Health Teacher Placement Standards and the Problems in the Placement Policy (보건교사 배치기준의 변천과정 및 배치정책의 문제 연구)

  • Kim, MiKyong
    • Journal of the Korean Society of School Health
    • /
    • v.26 no.3
    • /
    • pp.133-143
    • /
    • 2013
  • Purpose: The purpose of this study is to provide basic data for a more reasonable health teacher placement policy sending teachers to more appropriate sites, by analyzing the change process of the health teacher placement standards and the problems caused by an unreasonable placement policy. Methods: This study mainly analyzed relevant research data and existing studies focusing on a literature analysis. Results: To date, the placement policy for health teachers has changed, going through expansion, reduction, and retrogression, since its establishment. The standard, placing health teachers only in elementary schools with more than 18 classes, was created in 1952. Despite the expansion of the role of health teachers and the revision of the school health law in 2007, this standard has been applied to date without modification. In the meantime, there have been many problems caused by inappropriate placement of health teachers. It was difficult for health teachers in large schools to carry out proper health education; and, in many schools, passive health management, such as first aid, health tests, and student health management, was mainly executed rather than active health management. Students in small schools were not even given an opportunity to receive health education and health management owing to the absence of health teachers. Also, compared to teachers teaching other subjects, health teachers have had very unfair placement standards. Conclusion: The placement policy for health teachers, which has been applied to the present, has never reflected social change, the increase of student health issues, and the demand from the school area. Although the role of health teachers expanded with the execution of health education, the current placement standards for health teachers are very unreasonable. Accordingly, it is necessary to review the health teacher placement policy in a reasonable manner and to revise the standards considering the reality.

  • PDF

Scalable Service Placement in the Fog Computing Environment for the IoT-Based Smart City

  • Choi, Jonghwa;Ahn, Sanghyun
    • Journal of Information Processing Systems
    • /
    • v.15 no.2
    • /
    • pp.440-448
    • /
    • 2019
  • The Internet of Things (IoT) is one of the main enablers for situation awareness needed in accomplishing smart cities. IoT devices, especially for monitoring purposes, have stringent timing requirements which may not be met by cloud computing. This deficiency of cloud computing can be overcome by fog computing for which fog nodes are placed close to IoT devices. Because of low capabilities of fog nodes compared to cloud data centers, fog nodes may not be deployed with all the services required by IoT devices. Thus, in this article, we focus on the issue of fog service placement and present the recent research trends in this issue. Most of the literature on fog service placement deals with determining an appropriate fog node satisfying the various requirements like delay from the perspective of one or more service requests. In this article, we aim to effectively place fog services in accordance with the pre-obtained service demands, which may have been collected during the prior time interval, instead of on-demand service placement for one or more service requests. The concept of the logical fog network is newly presented for the sake of the scalability of fog service placement in a large-scale smart city. The logical fog network is formed in a tree topology rooted at the cloud data center. Based on the logical fog network, a service placement approach is proposed so that services can be placed on fog nodes in a resource-effective way.

A Network Load Sensitive Block Placement Strategy of HDFS

  • Meng, Lingjun;Zhao, Wentao;Zhao, Haohao;Ding, Yang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.9 no.9
    • /
    • pp.3539-3558
    • /
    • 2015
  • This paper investigates and analyzes the default block placement strategy of HDFS. HDFS is a typical representative distributed file system to stream vast amount of data effectively at high bandwidth to user applications. However, the default HDFS block placement policy assumes that all nodes in the cluster are homogeneous, and places blocks with a simple RoundRobin strategy without considering any nodes' resource characteristics, which decreases self-adaptability of the system. The primary contribution of this paper is the proposition of a network load sensitive block placement strategy. We have implemented our algorithm and justify it through extensive simulations and comparison with similar existing studies. The results indicate that our work not only performs much better in the data distribution but also improves write performance more significantly than the others.

Performance Optimization of Big Data Center Processing System - Big Data Analysis Algorithm Based on Location Awareness

  • Zhao, Wen-Xuan;Min, Byung-Won
    • International Journal of Contents
    • /
    • v.17 no.3
    • /
    • pp.74-83
    • /
    • 2021
  • A location-aware algorithm is proposed in this study to optimize the system performance of distributed systems for processing big data with low data reliability and application performance. Compared with previous algorithms, the location-aware data block placement algorithm uses data block placement and node data recovery strategies to improve data application performance and reliability. Simulation and actual cluster tests showed that the location-aware placement algorithm proposed in this study could greatly improve data reliability and shorten the application processing time of I/O interfaces in real-time.