• Journal of KIISE
• /
• v.43 no.6
• /
• pp.613-620
• /
• 2016

This paper presents a MOHA(Many-Task Computing on Hadoop) framework which aims to effectively apply the Many-Task Computing(MTC) technologies originally developed for high-performance processing of many tasks, to the existing Big Data processing platform Hadoop. We present basic concepts, motivation, preliminary results of PoC based on distributed message queue, and future research directions of MOHA. MTC applications may have relatively low I/O requirements per task. However, a very large number of tasks should be efficiently processed with potentially heavy inter-communications based on files. Therefore, MTC applications can show another pattern of data-intensive workloads compared to existing Hadoop applications, typically based on relatively large data block sizes. Through an effective convergence of MTC and Big Data technologies, we can introduce a new MOHA framework which can support the large-scale scientific applications along with the Hadoop ecosystem, which is evolving into a multi-application platform.

• Journal of Broadcast Engineering
• /
• v.24 no.6
• /
• pp.1122-1133
• /
• 2019

We have designed and implemented a new data processing framework called MOHA(Mtc On HAdoop) which can effectively support Many-Task Computing(MTC) applications in a YARN-based Hadoop platform. MTC applications can be composed of a very large number of computational tasks ranging from hundreds of thousands to millions of tasks, and each MTC application may have different resource usage patterns. Therefore, we have implemented MOHA-TaskExecutor(a pilot-job that executes real MTC application tasks)'s Adaptive Parallel Computability which can adaptively execute multiple tasks simultaneously, in order to improve the parallel computability of a YARN container and the overall system throughput. We have implemented multi-threaded version of TaskExecutor which can "independently and dynamically" adjust the number of concurrently running tasks, and in order to find the optimal number of concurrent tasks, we have employed Hill-Climbing algorithm.

• Journal of KIISE
• /
• v.42 no.1
• /
• pp.15-22
• /
• 2015

In this paper, we present a case study for a drug repositioning simulation based on distributed supercomputing technology that requires highly efficient processing of large-scale computations. Drug repositioning is the application of known drugs and compounds to new indications (i.e., new diseases), and this process requires efficient processing of a large number of docking tasks with relatively short per-task execution times. This mechanism shows the main characteristics of a Many-Task Computing (MTC) application, and as a representative case of MTC applications, we have applied a drug repositioning simulation in our HTCaaS system which can leverage distributed supercomputing infrastructure, and show that efficient task dispatching, dynamic resource allocation and load balancing, reliability, and seamless integration of multiple computing resources are crucial to support these challenging scientific applications.

• KIPS Transactions on Computer and Communication Systems
• /
• v.6 no.10
• /
• pp.407-414
• /
• 2017

Pig monitoring applications consisting of many tasks can take advantage of inherent data parallelism and enable parallel processing using performance accelerators. In this paper, we propose a task distribution method for pig monitoring applications into a heterogenous computing platform consisting of a multicore-CPU and a manycore-GPU. That is, a parallel program written in OpenCL is developed, and then the most suitable processor is determined based on the measured execution time of each task. The proposed method is simple but very effective, and can be applied to parallelize other applications consisting of many tasks on a heterogeneous computing platform consisting of a CPU and a GPU. Experimental results show that the performance of the proposed task distribution method on three different heterogeneous computing platforms can improve the performance of the typical GPU-only method where every tasks are executed on a deviceGPU by a factor of 1.5, 8.7 and 2.7, respectively.

• The Korean Journal of Applied Statistics
• /
• v.33 no.1
• /
• pp.87-105
• /
• 2020

Statistical engineering contains the design of experiments, quality control/management, and reliability engineering. R is a free software environment for statistical computing and graphics that is supported by the R Foundation for Statistical Computing. R package has many functions and libraries for statistical engineering. We can use R package as a useful tool for statistical engineering. This paper shows the applications of R package for statistical engineering and suggests a R Task View for statistical engineering.

• Broadcasting and Media Magazine
• /
• v.22 no.4
• /
• pp.11-17
• /
• 2017

본 고에서는 멀티 응용 데이터 플랫폼으로 진화하고 있는 하둡(Hadoop) 2.0의 주요 특징과 관련된 다양한 데이터 처리 프레임워크들에 대해서 기술하고자 한다. 기존의 맵리듀스(MapReduce) 기반의 배치 처리(Batch Processing)에 최적화되어 있던 하둡 1.0과는 달리, YARN의 등장과 함께 시작된 하둡 2.0 플랫폼은 다양한 형태의 데이터 처리 워크플로우들(Batch, Interactive, Streaming 등)을 지원할 수 있는 기능을 제공하고 있다. 또한, 최근에는 고성능컴퓨팅 분야에서 주로 활용되던 기술들도 하둡 2.0 플랫폼에서 지원되고 있다. 마지막으로 YARN 어플리케이션 개발 사례로서 본 연구팀에서 개발 중에 있는 Many-Task Computing (MTC) 응용을 위한 신규 데이터 처리 프레임워크를 소개한다.

• KIISE Transactions on Computing Practices
• /
• v.23 no.6
• /
• pp.392-396
• /
• 2017

Although mobile device-related technologies have developed rapidly, many problems arising from resource constraints have not been solved. Computation offloading that uses resources of cloud servers over the Internet was proposed to overcome physical limitations, and many studies have been conducted in terms of energy saving. However, completing tasks within their deadlines is more important than saving energy in real-time applications. In this paper, we proposed an offloading decision scheme considering the scheduling latency in the cloud to support real-time applications. The proposed scheme can improve the reliability of real-time tasks by comparing the estimated laxity of offloading a task with the estimated laxity of executing a task in a mobile device and selecting a more effective way to satisfy the task's deadline.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.15 no.3
• /
• pp.137-141
• /
• 2015

Mobile cloud computing uses cloud service to solve the resource constraint problem of mobile devices. Offloading means that a task executed on the mobile device commits to cloud and many studies related to the energy consumption have been researched. In this paper, we designed a response time model considering sensing data synchronization to estimate the efficiency of the offloading scheme in terms of the response time. The proposed model considers synchronization of required sensing data to improve the accuracy of response time estimation when cloud processes the task requested from a mobile device. We found that the response time is effected by new sensing data generation rate and synchronization period through simulation results.

• The Journal of the Convergence on Culture Technology
• /
• v.7 no.1
• /
• pp.615-619
• /
• 2021

This study focuses on memory systems that are optimized to increase performance and energy efficiency in many embedded systems such as IoT, cloud computing, and edge computing, and proposes a performance improvement technique. The proposed technique improves memory system performance based on machine learning algorithms that are widely used in many applications. The machine learning technique can be used for various applications through supervised learning, and can be applied to a data classification task used in improving memory system performance. Data classification based on highly accurate machine learning techniques enables data to be appropriately arranged according to data usage patterns, thereby improving overall system performance.

• Journal of applied mathematics & informatics
• /
• v.28 no.1_2
• /
• pp.351-361
• /
• 2010

The solution of large sparse linear systems is one of the most important problems in large scale scientific computing. Among the many methods developed, the preconditioned Krylov subspace methods are considered the preferred methods. Selecting a suitable preconditioner with appropriate parameters for a specific sparse linear system presents a challenging task for many application scientists and engineers who have little knowledge of preconditioned iterative methods. The prediction of ILU type preconditioners was considered in [27] where support vector machine(SVM), as a data mining technique, is used to classify large sparse linear systems and predict best preconditioners. In this paper, we apply the data mining approach to the sparse approximate inverse(SAI) type preconditioners to find some parameters with which the preconditioned Krylov subspace method on the linear systems shows best performance.