• Title/Summary/Keyword: Cluster computing environment

Search Result 109, Processing Time 0.021 seconds

Efficient Forwarding Path Computing Method for Context-Awareness Mobility Prediction Model (상황인식 이동성 예측 모델에서의 효율적인 포워딩 경로 산출 기법)

  • Jeong, Rae-jin;Oh, Young-jun;Lee, Kang-whan
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2014.10a
    • /
    • pp.93-95
    • /
    • 2014
  • In this paper, we proposed efficient forwarding path computing method using Context-Awareness Mobility Prediction Model. Context-Awareness Mobility Prediction Model is storing and classifying node's previous velocity and direction according to time in the hierarchical cluster structure. To overcome environment which node-to-node connection is broken off easily, the proposed algorithm calculate the connectivity formed matrix structure by comparing predicted velocity and direction, and use masking operation for selecting relay moving to destination. The proposed algorithm identified to show short delay by utilizing forwarding path which is continue node-to-node connection in the unstable situation.

  • PDF

Parallel Algorithm of Improved FunkSVD Based on Spark

  • Yue, Xiaochen;Liu, Qicheng
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.5
    • /
    • pp.1649-1665
    • /
    • 2021
  • In view of the low accuracy of the traditional FunkSVD algorithm, and in order to improve the computational efficiency of the algorithm, this paper proposes a parallel algorithm of improved FunkSVD based on Spark (SP-FD). Using RMSProp algorithm to improve the traditional FunkSVD algorithm. The improved FunkSVD algorithm can not only solve the problem of decreased accuracy caused by iterative oscillations but also alleviate the impact of data sparseness on the accuracy of the algorithm, thereby achieving the effect of improving the accuracy of the algorithm. And using the Spark big data computing framework to realize the parallelization of the improved algorithm, to use RDD for iterative calculation, and to store calculation data in the iterative process in distributed memory to speed up the iteration. The Cartesian product operation in the improved FunkSVD algorithm is divided into blocks to realize parallel calculation, thereby improving the calculation speed of the algorithm. Experiments on three standard data sets in terms of accuracy, execution time, and speedup show that the SP-FD algorithm not only improves the recommendation accuracy, shortens the calculation interval compared to the traditional FunkSVD and several other algorithms but also shows good parallel performance in a cluster environment with multiple nodes. The analysis of experimental results shows that the SP-FD algorithm improves the accuracy and parallel computing capability of the algorithm, which is better than the traditional FunkSVD algorithm.

Multi-platform Visualization System for Earth Environment Data (지구환경 데이터를 위한 멀티플랫폼 가시화 시스템)

  • Jeong, Seokcheol;Jung, Seowon;Kim, Jongyong;Park, Sanghun
    • Journal of the Korea Computer Graphics Society
    • /
    • v.21 no.3
    • /
    • pp.36-45
    • /
    • 2015
  • It is important subject of research in engineering and natural science field that creating continuing high-definition image from very large volume data. The necessity of software that helps analyze useful information in data has improved by effectively showing visual image information of high resolution data with visualization technique. In this paper, we designed multi-platform visualization system based on client-server to analyze and express earth environment data effectively constructed with observation and prediction. The visualization server comprised of cluster transfers data to clients through parallel/distributed computing, and the client is developed to be operated in various platform and visualize data. In addition, we aim user-friendly program through multi-touch, sensor and have made realistic simulation image with image-based lighting technique.

Design and Implementation of Incremental Learning Technology for Big Data Mining

  • Min, Byung-Won;Oh, Yong-Sun
    • International Journal of Contents
    • /
    • v.15 no.3
    • /
    • pp.32-38
    • /
    • 2019
  • We usually suffer from difficulties in treating or managing Big Data generated from various digital media and/or sensors using traditional mining techniques. Additionally, there are many problems relative to the lack of memory and the burden of the learning curve, etc. in an increasing capacity of large volumes of text when new data are continuously accumulated because we ineffectively analyze total data including data previously analyzed and collected. In this paper, we propose a general-purpose classifier and its structure to solve these problems. We depart from the current feature-reduction methods and introduce a new scheme that only adopts changed elements when new features are partially accumulated in this free-style learning environment. The incremental learning module built from a gradually progressive formation learns only changed parts of data without any re-processing of current accumulations while traditional methods re-learn total data for every adding or changing of data. Additionally, users can freely merge new data with previous data throughout the resource management procedure whenever re-learning is needed. At the end of this paper, we confirm a good performance of this method in data processing based on the Big Data environment throughout an analysis because of its learning efficiency. Also, comparing this algorithm with those of NB and SVM, we can achieve an accuracy of approximately 95% in all three models. We expect that our method will be a viable substitute for high performance and accuracy relative to large computing systems for Big Data analysis using a PC cluster environment.

Parallel Performance of Preconditioned Navier-Stokes Code on Myrinet Environment (Myrinet 환경에서 예조건화 Navier-Stokes 코드의 병렬처리 성능)

  • Kim M.-H.;Lee G. S.;Choi J.-Y.;Kim K. S.;Kim S.-L.;Jeung I.-S.
    • 한국전산유체공학회:학술대회논문집
    • /
    • 2001.05a
    • /
    • pp.149-154
    • /
    • 2001
  • Parallel performance of a Myrinet based PC-cluster was tested and compared with a conventional Fast-Ethernet system. A preconditioned Navier-Stokes code was parallelized with domain decomposition technique, and used for the parallel performance test. Speed-up ratio was examined as a major performance parameter depending on the number of processor and the network topology. As was expected, Myrinet system shows a superior parallel performance to the Fast-Ethernet system even with a single network adpater for a dual processor SMP machine. A test for the dependency on problem size also shows that network communication speed is a crucial factor for parallelized computational fluid dynamics analysis and the Myrinet system is a plausible candidate for high performance parallel computing system.

  • PDF

An Optimization of Resource Scheduling in the Cluster Computing Environment (클러스터 컴퓨팅 환경에서 자원 스케줄링의 최적화)

  • Lee, Hun Soon;Kim, Chang Soo
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2016.10a
    • /
    • pp.56-57
    • /
    • 2016
  • 본 논문에서는 클러스터 컴퓨팅 환경에서 가용한 자원의 추이를 고려한 자원 스케줄링 최적화 방법을 제안한다. 제안하는 스케줄링 방법은 데이터 인텐시브 응용의 특성인 자원 추가에 따른 처리 성능의 선형적 확장성을 활용하는데, 사용자가 작업 수행을 위해 명시한 자원이 가용하지 않은 경우에 해당 자원이 가용해질 때까지 기다리는 것이 아니라 가용한 자원 상황 추이를 고려하여 융통성 있는 자원 할당을 하게 함으로써 자원 조각화를 최소화한다. 시물레이션을 통한 실험으로 제안하는 방법이 기존 방법에 비해 자원 활용률과 처리량 측면에서 우수함을 검증하였다.

PHLOX : A High Performance Transaction Processing System on Workstation Cluster Environment with Data Sharing (PHLOX : 데이터를 공유하는 워크스테이션 클러스터 환경기반 고성능 트랜잭션 처리 시스템)

  • Jo, Haeng-Rae;Mun, Ae-Gyeong;Jeong, Jae-Gyu;Park, Jae-Yong;Kim, Tae-U;Lee, U-Beom
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.5 no.1
    • /
    • pp.29-42
    • /
    • 1999
  • 은행이나 증권시장 혹은 통신 분야와 같은 온라인 트랜잭션처리 (OLTP) 응용 분야들이 계속 증가하고 있으며 그 규모와 복잡성은 매년 405에서 50% 이상 증가하고 있는 것으로 평가되고 있다. 이러한 복잡한 OLTP 응용분야들은 한 대의 개인용 컴퓨터나 워크스테이션으로는 처리가 불가능하며, 슈퍼 컴퓨터나 MPP와 같은 고성능의 컴퓨터를 이용하는 경우에는 방대한 예산이 소요될 것이다. 본 논문에서는 데이터를 공유하는 워크스테이션 클러스터에서 트랜잭션을 병렬로 처리할 수 있는 고성능 트랜잭션처리 시스템인 PHLOX를개발한다. PHLOX는 네트워크 파일 시스템을 이용하여 데이터를 공유함으로써 확장성이 뛰어나며 각 워크스테이션의 메모리 버퍼간에 직접 데이터 전송을 지원함으로써 디스크 액세스빈도수를 줄일 수 있다는 장점을 갖는다.

A Study of Stable Intrusion Detection for MANET (MANET에서 안정된 침입탐지에 관한 연구)

  • Yang, Hwan-Seok;Yang, Jeong-Mo
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.22 no.1
    • /
    • pp.93-98
    • /
    • 2012
  • MANET composed of only moving nodes is concerned to core technology to construct ubiquitous computing environment. Also, it is a lack of security because of no middle infrastructure. So, it is necessary to intrusion detection system which can track malicious attack. In this study, cluster was used to stable intrusion detection, and rule about various attacks was defined to detect accurately attack that seems like network problem. Proposed method through experience was confirmed that stable detection rate was showed although number of nodes increase.

A Dynamic Co-scheduling Scheme for MPI-based Parallel Programs on Linux Clusters (리눅스 클러스터에서 MPI 기반 병렬 프로그램의 동적 동시 스케줄링 기법)

  • Kim, Hyuk;Rhee, Yun-Seok
    • Journal of the Korea Society of Computer and Information
    • /
    • v.13 no.1
    • /
    • pp.29-35
    • /
    • 2008
  • For efficient message passing of Parallel programs, it is required to schedule the involved two processes at the same time which are executed on different nodes, that is called 'co-scheduling' However, each node of cluster systems is built on top of general purpose multitasking OS. which autonomously manages local Processes. Thus it is not so easy to co-schedule two (or more) processes in such computing environment. Our work proposes a co-scheduling scheme for MPI-based parallel programs which exploits message exchange information between two parties. We implement the scheme on Linux cluster which requires slight kernel hacking and MPI library modification. The experiment with NPB parallel suite shows that our scheme results in 33-56% reduction in the execution time compared to the typical scheduling case. and especially better Performance in more communication-bound applications.

  • PDF

Deployment and Performance Analysis of Data Transfer Node Cluster for HPC Environment (HPC 환경을 위한 데이터 전송 노드 클러스터 구축 및 성능분석)

  • Hong, Wontaek;An, Dosik;Lee, Jaekook;Moon, Jeonghoon;Seok, Woojin
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.9 no.9
    • /
    • pp.197-206
    • /
    • 2020
  • Collaborative research in science applications based on HPC service needs rapid transfers of massive data between research colleagues over wide area network. With regard to this requirement, researches on enhancing data transfer performance between major superfacilities in the U.S. have been conducted recently. In this paper, we deploy multiple data transfer nodes(DTNs) over high-speed science networks in order to move rapidly large amounts of data in the parallel filesystem of KISTI's Nurion supercomputer, and perform transfer experiments between endpoints with approximately 130ms round trip time. We have shown the results of transfer throughput in different size file sets and compared them. In addition, it has been confirmed that the DTN cluster with three nodes can provide about 1.8 and 2.7 times higher transfer throughput than a single node in two types of concurrency and parallelism settings.