• Title/Summary/Keyword: memory balancing

Search Result 61, Processing Time 0.032 seconds

A Parallel HDFS and MapReduce Functions for Emotion Analysis (감성분석을 위한 병렬적 HDFS와 맵리듀스 함수)

  • Back, BongHyun;Ryoo, Yun-Kyoo
    • Journal of the Korea society of information convergence
    • /
    • v.7 no.2
    • /
    • pp.49-57
    • /
    • 2014
  • Recently, opinion mining is introduced to extract useful information from SNS data and to evaluate the true intention of users. Opinion mining are required several efficient techniques to collect and analyze a large amount of SNS data and extract meaningful data from them. Therefore in this paper, we propose a parallel HDFS(Hadoop Distributed File System) and emotion functions based on Mapreduce to extract some emotional information of users from various unstructured big data on social networks. The experiment results have verified that the proposed system and functions perform faster than O(n) for data gathering time and loading time, and maintain stable load balancing for memory and CPU resources.

  • PDF

A Load Balancing Scheme for Distributed SDN Based on Harmony Search with K-means Clustering (K-means 군집화 및 Harmony Search 알고리즘을 이용한 분산 SDN의 부하 분산 기법)

  • Kim, Se-Jun;Yoo, Seung-Eon;Lee, Byung-Jun;Kim, Kyung-Tae;Youn, Hee-Yong
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2019.01a
    • /
    • pp.29-30
    • /
    • 2019
  • 본 논문에서는 다중 컨트롤러가 존재하는 분산 SDN 환경에서 과도한 제어 메시지로 인한 과부하된 컨트롤러의 부하를 줄이기 위하여 이주할 스위치를 K-means 군집화와 Harmony Search(HS)를 기반으로 선정 하는 기법을 제안하였다. 기존에 HS를 이용하여 이주할 스위치를 선택하는 기법이 제시되었으나, 시간 소모에 비하여 정확도가 부족한 단점이 있다. 또한 Harmony Memory(HM) 구축을 위해 메모리 소모 또한 크다. 이를 해결하기 위하여 본 논문에서는 유클리드 거리를 기반으로 하는 K-means 군집화를 이용하여 이주할 스위치를 골라내어 HM의 크기를 줄이고 이주 효율을 향상 시킨다.

  • PDF

PESA: Prioritized experience replay for parallel hybrid evolutionary and swarm algorithms - Application to nuclear fuel

  • Radaideh, Majdi I.;Shirvan, Koroush
    • Nuclear Engineering and Technology
    • /
    • v.54 no.10
    • /
    • pp.3864-3877
    • /
    • 2022
  • We propose a new approach called PESA (Prioritized replay Evolutionary and Swarm Algorithms) combining prioritized replay of reinforcement learning with hybrid evolutionary algorithms. PESA hybridizes different evolutionary and swarm algorithms such as particle swarm optimization, evolution strategies, simulated annealing, and differential evolution, with a modular approach to account for other algorithms. PESA hybridizes three algorithms by storing their solutions in a shared replay memory, then applying prioritized replay to redistribute data between the integral algorithms in frequent form based on their fitness and priority values, which significantly enhances sample diversity and algorithm exploration. Additionally, greedy replay is used implicitly to improve PESA exploitation close to the end of evolution. PESA features in balancing exploration and exploitation during search and the parallel computing result in an agnostic excellent performance over a wide range of experiments and problems presented in this work. PESA also shows very good scalability with number of processors in solving an expensive problem of optimizing nuclear fuel in nuclear power plants. PESA's competitive performance and modularity over all experiments allow it to join the family of evolutionary algorithms as a new hybrid algorithm; unleashing the power of parallel computing for expensive optimization.

Implementation of Multicore-Aware Load Balancing on Clusters through Data Distribution in Chapel (클러스터 상에서 다중 코어 인지 부하 균등화를 위한 Chapel 데이터 분산 구현)

  • Gu, Bon-Gen;Carpenter, Patrick;Yu, Weikuan
    • The KIPS Transactions:PartA
    • /
    • v.19A no.3
    • /
    • pp.129-138
    • /
    • 2012
  • In distributed memory architectures like clusters, each node stores a portion of data. How data is distributed across nodes influences the performance of such systems. The data distribution scheme is the strategy to distribute data across nodes and realize parallel data processing. Due to various reasons such as maintenance, scale up, upgrade, etc., the performance of nodes in a cluster can often become non-identical. In such clusters, data distribution without considering performance cannot efficiently distribute data on nodes. In this paper, we propose a new data distribution scheme based on the number of cores in nodes. We use the number of cores as the performance factor. In our data distribution scheme, each node is allocated an amount of data proportional to the number of cores in it. We implement our data distribution scheme using the Chapel language. To show our data distribution is effective in reducing the execution time of parallel applications, we implement Mandelbrot Set and ${\pi}$-Calculation programs with our data distribution scheme, and compare the execution times on a cluster. Based on experimental results on clusters of 8-core and 16-core nodes, we demonstrate that data distribution based on the number of cores can contribute to a reduction in the execution times of parallel programs on clusters.

Development of Intelligent Load Balancing Algorithm in Application of Fuzzy-Neural Network (퍼지-뉴럴 네트워크를 응용한 지능형 로드밸런싱 알고리즘 개발)

  • Chu, Gyo-Soo;Kim, Wan-Yong;Jung, Jae-Yun;Kim, Hag-Bae
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.30 no.2B
    • /
    • pp.36-43
    • /
    • 2005
  • This paper suggests a method to effectively apply an application model of fuzzy-neural network to the optimal load distribution algorithm, considering the complication and non-linearity of the web server environment. We use the clustering web server in the linux system and it consists of a load balancer that distributes the network loads and some of real servers that processes the load and responses to the client. The previous works considered only with the scrappy decision information such as the connections. That is, since the distribution algorithm depends on the input of the whole network throughput, it was proved inefficient in terms of performance improvement of the web server. With the proposed algorithm, it monitors the whole states of both network input and output. Then, it infers CPU and memory states of each real server and effectively distributes the requests of the clients. In this paper, the proposed model is compared with the previous method through simulations and we analysis the results to develop the optimal and intelligent load balancing model.

Design and Implementation of Multilingual support method for 3-tiered softwares (3-TIER 구조 소프트웨어의 다국어 지원 방식의 설계와 구현)

  • Koh, Jeong-Gook
    • Journal of Korea Multimedia Society
    • /
    • v.15 no.2
    • /
    • pp.266-272
    • /
    • 2012
  • Multilingual support of software is necessary for entering global market. 3-tier architecture is a solution for problems of 2-tier architecture. It divides an application into a client-tier and an application-tier, and presentation logic and database are connected by middleware. The advantage of 3-tier architecture is the enhanced performance through load balancing, scalability, easier maintenance and reusability. This paper proposes a multilingual support method that utilizes common resource files for 3-tier enterprise softwares, applies the proposed method to development of multilingual version of billing solution, and verify the usefulness of it. It is easy for development and maintenance of software, the addition of language supported. Proposed method holds a resource file for each language and provides a multilingual support class library. Therefore this method reduces a waste of memory and disk space. Deployment of a class library into an application tier makes development and maintenance of software, the addition of new language supported easy. To avoid inappropriate modification of a resource file, a multilingual support class library is provided in a dll file.

Communication Schedule for GEN_BLOCK Redistribution (GEN_BLOCK간 재분산을 위한 통신 스케줄)

  • Yook, Hyun-Gyoo;Park, Myong-Soon
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.27 no.5
    • /
    • pp.450-463
    • /
    • 2000
  • Array redistribution is usually required to enhance algorithm performance in many parallel programs on distributed memory multicomputers. GEN_BLOCK redistribution, which is redistribution between different GEN_BLOCKs, is essential for load balancing. However, prior research on redistribution has been focused on regular redistribution, such as redistribution between different CYCLIC(N)s. GEN_BLOCK redistribution is very different from regular redistribution. Message passing in regular redistribution involves repetitions of basic message passing patterns, while message passing for GEN_BLOCK redistribution shows locality. This paper proves that two optimal condition, reducing the number of communication steps and minimizing redistribution size, are essential in GEN_BLOCK redistribution. Additionally, by adding a relocation phase to list scheduling, we make an optimal scheduling algorithm for GEN_BLOCK redistribution. To evaluate the performance of the algorithm, we have performed experiments on a CRAY T3E. According to the experiments, it was proven that the scheduling algorithm shows better performance and that the conditions are critical in enhancing the communication speed of GEN_BLOCK redistribution.

  • PDF

Energy/Distance Estimation-based and Distributed Selection/Migration of Cluster Heads in Wireless Sensor Networks (센서 네트워크의 에너지 및 거리 추정 기반 분산 클러스터 헤드 선정과 이주 방법)

  • Kim, Dong-Woo;Park, Jong-Ho;Lee, Tae-Jin
    • Journal of the Institute of Electronics Engineers of Korea TC
    • /
    • v.44 no.3 s.357
    • /
    • pp.18-25
    • /
    • 2007
  • In sensor networks, sensor nodes have limited computational capacity, power and memory. Thus energy efficiency is one of the most important requirements. How to extend the lifetime of wireless sensor networks has been widely discussed in recent years. However, one of the most effective approaches to cope with power conservation, network scalability, and load balancing is clustering technique. The function of a cluster head is to collect and route messages of all the nodes within its cluster. Cluster heads must be changed periodically for low energy consumption and load distribution. In this paper, we propose an energy-aware cluster head selection algorithm and Distance Estimation-based distributed Clustering Algorithm (DECA) in wireless sensor networks, which exchanges cluster heads for less energy consumption by distance estimation. Our simulation result shows that DECA can improve the system lifetime of sensor networks up to three times compared to the conventional scheme.

Efficient Load Balancing Scheme using Resource Information in Web Server System (웹 서버 시스템에서의 자원 정보를 이용한 효율적인 부하분산 기법)

  • Chang Tae-Mu;Myung Won-Shig;Han Jun-Tak
    • The KIPS Transactions:PartA
    • /
    • v.12A no.2 s.92
    • /
    • pp.151-160
    • /
    • 2005
  • The exponential growth of Web users requires the web serves with high expandability and reliability. It leads to the excessive transmission traffic and system overload problems. To solve these problems, cluster systems are widely studied. In conventional cluster systems, when the request size is large owing to such types as multimedia and CGI, the particular server load and response time tend to increase even if the overall loads are distributed evenly. In this paper, a cluster system is proposed where each Web server in the system has different contents and loads are distributed efficiently using the Web server resource information such as CPU, memory and disk utilization. Web servers having different contents are mutually connected and managed with a network file system to maintain information consistency required to support resource information updates, deletions, and additions. Load unbalance among contents group owing to distribution of contents can be alleviated by reassignment of Web servers. Using a simulation method, we showed that our method shows up to $50\%$ about average throughput and processing time improvement comparing to systems using each LC method and RR method.

Implementation of a Cluster VOD Server and an Embedded Client based on Linux (리눅스 기반의 클러스터 VOD서버와 내장형에 클라이언트의 구현)

  • Seo Dongmahn;Bang Cheolseok;Lee Joahyoung;Kim Byounggil;Jung Inbum
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.10 no.6
    • /
    • pp.435-447
    • /
    • 2004
  • For VOD systems, it is important to provide QoS to more users under the limited resources. To analyze QoS issues in real environment, we implement clustered VOD server and embedded client system based on the Linux open source platform. The parallel processing of MPEG data, load balancing for nodes and VCR like functions are implemented in the server side. To provide more user friendly interface, the general TV is used for a VOD client's terminal and the embedded board is used supporting for VCR functions. In this paper, we measure the performance of the implemented VOD system under the various user requirement features and evaluate the sources of performance limitations. From these analyses, we propose the dynamic admission control method based on the availability memory and network bandwidth. The proposed method enhances the utilization of the system resource for the more QoS media streams.