• Title/Summary/Keyword: Cluster computing environment

Search Result 109, Processing Time 0.024 seconds

An Improved Estimation Model of Server Power Consumption for Saving Energy in a Server Cluster Environment (서버 클러스터 환경에서 에너지 절약을 위한 향상된 서버 전력 소비 추정 모델)

  • Kim, Dong-Jun;Kwak, Hu-Keun;Kwon, Hui-Ung;Kim, Young-Jong;Chung, Kyu-Sik
    • The KIPS Transactions:PartA
    • /
    • v.19A no.3
    • /
    • pp.139-146
    • /
    • 2012
  • In the server cluster environment, one of the ways saving energy is to control server's power according to traffic conditions. This is to determine the ON/OFF state of servers according to energy usage of data center and each server. To do this, we need a way to estimate each server's energy. In this paper, we use a software-based power consumption estimation model because it is more efficient than the hardware model using power meter in terms of energy and cost. The traditional software-based power consumption estimation model has a drawback in that it doesn't know well the computing status of servers because it uses only the idle status field of CPU. Therefore it doesn't estimate consumption power effectively. In this paper, we present a CPU field based power consumption estimation model to estimate more accurate than the two traditional models (CPU/Disk/Memory utilization based power consumption estimation model and CPU idle utilization based power consumption estimation model) by using the various status fields of CPU to get the CPU status of servers and the overall status of system. We performed experiments using 2 PCs and compared the power consumption estimated by the power consumption model (software) with that measured by the power meter (hardware). The experimental results show that the traditional model has about 8-15% average error rate but our proposed model has about 2% average error rate.

Unmanned Aircraft Platform Based Real-time LiDAR Data Processing Architecture for Real-time Detection Information (실시간 탐지정보 제공을 위한 무인기 플랫폼 기반 실시간 LiDAR 데이터 처리구조)

  • Eum, Junho;Berhanu, Eyassu;Oh, Sangyoon
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.12
    • /
    • pp.745-750
    • /
    • 2015
  • LiDAR(Light Detection and Ranging) technology provides realistic 3-dimension image information, and it has been widely utilized in various fields. However, the utilization of this technology in the military domain requires prompt responses to dynamically changing tactical environment and is therefore limited since LiDAR technology requires complex processing in order for extensive amounts of LiDAR data to be utilized. In this paper, we introduce an Unmanned Aircraft Platform Based Real-time LiDAR Data Processing Architecture that can provide real-time detection information by parallel processing and off-loading between the UAV processing and high-performance data processing areas. We also conducted experiments to verify the feasibility of our proposed architecture. Processing with ARM cluster similar to the UAV platform processing area results in similar or better performance when compared to the current method. We determined that our proposed architecture can be utilized in the military domain for tactical and combat purposes such as unmanned monitoring system.

Development of IoT Service Classification Method based on Service Operation Characteristic (세부 동작 기반 사물인터넷 서비스 분류 기법 개발)

  • Jo, Jeong hoon;Lee, HwaMin;Lee, Dae won
    • Journal of Internet Computing and Services
    • /
    • v.19 no.2
    • /
    • pp.17-26
    • /
    • 2018
  • Recently, through the emergence and convergence of Internet services, the unified Internet of thing(IoT) service platform have been researched. Currently, the IoT service is constructed as an independent system according to the purpose of the service provider, so information exchange and module reuse are impossible among similar services. In this paper, we propose a operation based service classification algorithm for various services in order to provide an environment of unfied Internet platform. In implementation, we classify and cluster more than 100 commercial IoT services. Based on this, we evaluated the performance of the proposed algorithm compared with the K-means algorithm. In order to prevent a single clustering due to the lack of sample groups, we re-cluster them using K-means algorithm. In future study, we will expand existing service sample groups and use the currently implemented classification system on Apache Spark for faster and more massive data processing.

Implementation of a Cluster VOD Server and an Embedded Client based on Linux (리눅스 기반의 클러스터 VOD서버와 내장형에 클라이언트의 구현)

  • Seo Dongmahn;Bang Cheolseok;Lee Joahyoung;Kim Byounggil;Jung Inbum
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.10 no.6
    • /
    • pp.435-447
    • /
    • 2004
  • For VOD systems, it is important to provide QoS to more users under the limited resources. To analyze QoS issues in real environment, we implement clustered VOD server and embedded client system based on the Linux open source platform. The parallel processing of MPEG data, load balancing for nodes and VCR like functions are implemented in the server side. To provide more user friendly interface, the general TV is used for a VOD client's terminal and the embedded board is used supporting for VCR functions. In this paper, we measure the performance of the implemented VOD system under the various user requirement features and evaluate the sources of performance limitations. From these analyses, we propose the dynamic admission control method based on the availability memory and network bandwidth. The proposed method enhances the utilization of the system resource for the more QoS media streams.

A Global Buffer Manager for a Shared Disk File System in SAN Clusters (SAN 환경에서 공유 디스크 파일 시스템을 위한 전역 버퍼 관리자)

  • 박선영;손덕주;신범주;김학영;김명준
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.10 no.2
    • /
    • pp.134-145
    • /
    • 2004
  • With rapid growth in the amount of data transferred on the Internet, traditional storage systems have reached the limits of their capacity and performance. SAN (Storage Area Network), which connects hosts to disk with the Fibre Channel switches, provides one of the powerful solutions to scale the data storage and servers. In this environment, the maintenance of data consistency among hosts is an important issue because multiple hosts share the files on disks attached to the SAN. To preserve data consistency, each host can execute the disk I/O whenever disk read and write operations are requested. However, frequent disk I/O requests cause the deterioration of the overall performance of a SAN cluster. In this paper, we introduce a SANtopia global buffer manager to improve the performance of a SAN cluster reducing the number of disk I/Os. We describe the design and algorithms of the SANtopia global buffer manager, which provides a buffer cache sharing mechanism among the hosts in the SAN cluster. Micro-benchmark results to measure the performance of block I/O operations show that the global buffer manager achieves speed-up by the factor of 1.8-12.8 compared with the existing method using disk I/O operations. Also, File system micro-benchmark results show that SANtopia file system with the global buffer manager improves performance by the factor of 1.06 in case of directories and 1.14 in case of files compared with the file system without a global buffer manager.

Efficient QoS Policy Implementation Using DSCP Redefinition: Towards Network Load Balancing (DSCP 재정의를 통한 효율적인 QoS 정책 구현: 네트워크 부하 분산을 위해)

  • Hanwoo Lee;Suhwan Kim;Gunwoo Park
    • The Journal of the Convergence on Culture Technology
    • /
    • v.9 no.3
    • /
    • pp.715-720
    • /
    • 2023
  • The military is driving innovative changes such as AI, cloud computing, and drone operation through the Fourth Industrial Revolution. It is expected that such changes will lead to a rapid increase in the demand for information exchange requirements, reaching all lower-ranking soldiers, as networking based on IoT occurs. The flow of such information must ensure efficient information distribution through various infrastructures such as ground networks, stationary satellites, and low-earth orbit small communication satellites, and the demand for information exchange that is distributed through them must be appropriately dispersed. In this study, we redefined the DSCP, which is closely related to QoS (Quality of Service) in information dissemination, into 11 categories and performed research to map each cluster group identified by cluster analysis to the defense "information exchange requirement list" on a one-to-one basis. The purpose of the research is to ensure efficient information dissemination within a multi-layer integrated network (ground network, stationary satellite network, low-earth orbit small communication satellite network) with limited bandwidth by re-establishing QoS policies that prioritize important information exchange requirements so that they are routed in priority. In this paper, we evaluated how well the information exchange requirement lists classified by cluster analysis were assigned to DSCP through M&S, and confirmed that reclassifying DSCP can lead to more efficient information distribution in a network environment with limited bandwidth.

Clustering Algorithm for Extending Lifetime of Wireless Sensor Networks (무선 센서 네트워크의 수명연장을 위한 클러스터링 알고리즘)

  • Kim, Sun-Chol;Choi, Seung-Kwon;Cho, Yong-Hwan
    • Journal of the Korea Society of Computer and Information
    • /
    • v.20 no.4
    • /
    • pp.77-85
    • /
    • 2015
  • Recently, wireless sensor network(WSN) have been used in various fields to implement ubiquitous computing environment. WSN uses small, low cost and low power sensors in order to collect information from the sensor field. This paper proposes a clustering algorithm for energy efficiency of sensor nodes. The proposed algorithm is based on conventional LEACH, the representative clustering protocol for WSN and it prolongs network and nodes life time using sleep technique and changable transmission mode. The nodes of the proposed algorithm first calculate their clustering participation value based on the distance to the neighbor nodes. The nodes located in high density area will have clustering participation value and it can turn to sleep mode. Besides, proposed algorithm can change transmission method from conventional single-hop transmission to multi-hop transmission according to the energy level of cluster head. Simulation results show that the proposed clustering algorithm outperforms conventional LEACH, especially non-uniformly deployed network.

Distributed data deduplication technique using similarity based clustering and multi-layer bloom filter (SDS 환경의 유사도 기반 클러스터링 및 다중 계층 블룸필터를 활용한 분산 중복제거 기법)

  • Yoon, Dabin;Kim, Deok-Hwan
    • The Journal of Korean Institute of Next Generation Computing
    • /
    • v.14 no.5
    • /
    • pp.60-70
    • /
    • 2018
  • A software defined storage (SDS) is being deployed in cloud environment to allow multiple users to virtualize physical servers, but a solution for optimizing space efficiency with limited physical resources is needed. In the conventional data deduplication system, it is difficult to deduplicate redundant data uploaded to distributed storages. In this paper, we propose a distributed deduplication method using similarity-based clustering and multi-layer bloom filter. Rabin hash is applied to determine the degree of similarity between virtual machine servers and cluster similar virtual machines. Therefore, it improves the performance compared to deduplication efficiency for individual storage nodes. In addition, a multi-layer bloom filter incorporated into the deduplication process to shorten processing time by reducing the number of the false positives. Experimental results show that the proposed method improves the deduplication ratio by 9% compared to deduplication method using IP address based clusters without any difference in processing time.

Development of Retargetable Hadoop Simulation Environment Based on DEVS Formalism (DEVS 형식론 기반의 재겨냥성 하둡 시뮬레이션 환경 개발)

  • Kim, Byeong Soo;Kang, Bong Gu;Kim, Tag Gon;Song, Hae Sang
    • Journal of the Korea Society for Simulation
    • /
    • v.26 no.4
    • /
    • pp.51-61
    • /
    • 2017
  • Hadoop platform is a representative storing and managing platform for big data. Hadoop consists of distributed computing system called MapReduce and distributed file system called HDFS. It is important to analyse the effectiveness according to the change of cluster constructions and several parameters. However, since it is hard to construct thousands of clusters and analyse the constructed system, simulation method is required to analyse the system. This paper proposes Hadoop simulator based on DEVS formalism which provides hierarchical and modular modeling. Hadoop simulator provides a retargetable experimental environment that is possible to change of various parameters, algorithms and models. It is also possible to design input models reflecting the characteristics of Hadoop applications. To maximize the user's convenience, the user interface, real-time model viewer, and input scenario editor are also provided. In this paper, we validate Hadoop Simulator through the comparison with the Hadoop execution results and perform various experiments.

Job-aware Network Scheduling for Hadoop Cluster

  • Liu, Wen;Wang, Zhigang;Shen, Yanming
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.1
    • /
    • pp.237-252
    • /
    • 2017
  • In recent years, data centers have become the core infrastructure to deal with big data processing. For these big data applications, network transmission has become one of the most important factors affecting the performance. In order to improve network utilization and reduce job completion time, in this paper, by real-time monitoring from the application layer, we propose job-aware priority scheduling. Our approach takes the correlations of flows in the same job into account, and flows in the same job are assigned the same priority. Therefore, we expect that flows in the same job finish their transmissions at about the same time, avoiding lagging flows. To achieve load balancing, two approaches (Flow-based and Spray) using ECMP (Equal-Cost multi-path routing) are presented. We implemented our scheme using NS-2 simulator. In our evaluations, we emulate real network environment by setting background traffic, scheduling delay and link failures. The experimental results show that our approach can enhance the Hadoop job execution efficiency of the shuffle stage, significantly reduce the network transmission time of the highest priority job.