• Title/Summary/Keyword: Data-intensive processing

Search Result 131, Processing Time 0.029 seconds

Performance analysis of local exit for distributed deep neural networks over cloud and edge computing

  • Lee, Changsik;Hong, Seungwoo;Hong, Sungback;Kim, Taeyeon
    • ETRI Journal
    • /
    • v.42 no.5
    • /
    • pp.658-668
    • /
    • 2020
  • In edge computing, most procedures, including data collection, data processing, and service provision, are handled at edge nodes and not in the central cloud. This decreases the processing burden on the central cloud, enabling fast responses to end-device service requests in addition to reducing bandwidth consumption. However, edge nodes have restricted computing, storage, and energy resources to support computation-intensive tasks such as processing deep neural network (DNN) inference. In this study, we analyze the effect of models with single and multiple local exits on DNN inference in an edge-computing environment. Our test results show that a single-exit model performs better with respect to the number of local exited samples, inference accuracy, and inference latency than a multi-exit model at all exit points. These results signify that higher accuracy can be achieved with less computation when a single-exit model is adopted. In edge computing infrastructure, it is therefore more efficient to adopt a DNN model with only one or a few exit points to provide a fast and reliable inference service.

BenchGAD: An Integrated Testing Framework for GPU Accelerated Data-intensive Systems (BenchGAD: GPU 기반 데이터 집약 시스템의 효과적인 테스팅 프레임워크)

  • Gu, Sang-Un;Choi, Byeong-wook;Suh, Young-Kyoon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2018.05a
    • /
    • pp.317-319
    • /
    • 2018
  • 최근 발생하는 데이터는 대용량이며, 형태가 다양하고 빠르게 생성되는 특징이 있다. 이러한 데이터는 CPU, 인메모리 기반인 기존의 데이터 처리 시스템에서 처리하는데 많은 시간이 소모된다. 이 문제를 해결하기 위해 GPU 기반 데이터 집약 시스템이 출현하기 시작했다. 하지만, 이러한 시스템의 성능을 종합적으로 측정하는 테스트 결과는 시스템마다 다른 기준으로 제공하고 있다. 이에 따라, 개발자 및 사용자는 성능 병목 현상을 탐색하고 해결하는 데 큰 어려움을 겪을 수 있다. 즉, 이러한 다른 기준으로는 개발자 및 사용자가 시스템의 통합적인 성능 비교 분석을 수행하기 힘들다. 이러한 문제를 해결하기 위해서, 본 논문은 원스탑 테스팅 프레임워크인 BenchGAD 를 제안하고자한다.

A Model for Illegal File Access Tracking Using Windows Logs and Elastic Stack

  • Kim, Jisun;Jo, Eulhan;Lee, Sungwon;Cho, Taenam
    • Journal of Information Processing Systems
    • /
    • v.17 no.4
    • /
    • pp.772-786
    • /
    • 2021
  • The process of tracking suspicious behavior manually on a system and gathering evidence are labor-intensive, variable, and experience-dependent. The system logs are the most important sources for evidences in this process. However, in the Microsoft Windows operating system, the action events are irregular and the log structure is difficult to audit. In this paper, we propose a model that overcomes these problems and efficiently analyzes Microsoft Windows logs. The proposed model extracts lists of both common and key events from the Microsoft Windows logs to determine detailed actions. In addition, we show an approach based on the proposed model applied to track illegal file access. The proposed approach employs three-step tracking templates using Elastic Stack as well as key-event, common-event lists and identify event lists, which enables visualization of the data for analysis. Using the three-step model, analysts can adjust the depth of their analysis.

Accelerating 2D DCT in Multi-core and Many-core Environments (멀티코어와 매니코어 환경에서의 2 차원 DCT 가속)

  • Hong, Jin-Gun;Jung, Sung-Wook;Kim, Cheong-Ghil;Burgstaller, Bernd
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2011.04a
    • /
    • pp.250-253
    • /
    • 2011
  • Chip manufacture nowadays turned their attention from accelerating uniprocessors to integrating multiple cores on a chip. Moreover desktop graphic hardware is now starting to support general purpose computation. Desktop users are able to use multi-core CPU and GPU as a high performance computing resources these days. However exploiting parallel computing resources are still challenging because of lack of higher programming abstraction for parallel programming. The 2-dimensional discrete cosine transform (2D-DCT) algorithms are most computational intensive part of JPEG encoding. There are many fast 2D-DCT algorithms already studied. We implemented several algorithms and estimated its runtime on multi-core CPU and GPU environments. Experiments show that data parallelism can be fully exploited on CPU and GPU architecture. We expect parallelized DCT bring performance benefit towards its applications such as JPEG and MPEG.

Design and Implementation of Customized Farming Applications using Public Data (공공데이터를 이용한 맞춤형 영농 어플리케이션 설계 및 구현)

  • Ko, Jooyoung;Yoon, Sungwook;Kim, Hyenki
    • Journal of Korea Multimedia Society
    • /
    • v.18 no.6
    • /
    • pp.772-779
    • /
    • 2015
  • Advancing information technology have rapidly changed our service environment of life, culture, and industry. Computer information communication system is applied in medical, health, distribution, and business transaction. Smart is using new information by combining ability of computer and information. Although agriculture is labor intensive industry that requires a lot of hands, agriculture is becoming knowledge-based industry today. In agriculture field, computer communication system is applied on facilities farming and machinery Agricultural. In this paper, we designed and implemented application that provides personalized agriculture related information at the actual farming field. Also, this provides farmer a system that they can directly auction or sell their produced crops. We designed and implemented a system that parsing information of each seasonal, weather condition, market price, region based, crop, and disease and insects through individual setup on ubiquitous environment using location-based sensor network and processing data.

Model Coupling Technique for Level Access in Hierarchical Simulation Models and Its Applications (계층의 구조를 갖는 시뮬레이션 모델에 있어서 단계적 접근을 위한 모델연결 방법론과 그 적용 예)

  • 조대호
    • Journal of the Korea Society for Simulation
    • /
    • v.5 no.2
    • /
    • pp.25-40
    • /
    • 1996
  • Modeling of systems for intensive knowledge-based processing requires a modeling methodology that makes efficient access to the information in huge data base models. The proposed level access mothodology is a modeling approach applicable to systems where data is stored in a hierarchical and modular modules of active memory cells(processor/memory pairs). It significantly reduces the effort required to create discrete event simulation models constructed in hierarchical, modular fashion for above application. Level access mothodology achieves parallel access to models within the modular, hierarchical modules(clusters) by broadcasting the desired operations(e.g. querying information, storing data and so on) to all the cells below a certain desired hierarchical level. Level access methodology exploits the capabilities of object-oriented programming to provide a flexible communication paradigm that combines port-to-port coupling with name-directed massaging. Several examples are given to illustrate the utility of the methodology.

  • PDF

Application of a wireless pressure sensing system to coastal wind monitoring

  • Pinelli, J.P.;Subramanian, C.S.;Lapilli, C.;Buist, L.
    • Wind and Structures
    • /
    • v.8 no.3
    • /
    • pp.179-196
    • /
    • 2005
  • This paper describes the application of a wireless data acquisition system to monitor wind pressures and velocities with absolute pressure sensors and an anemometer. The system was developed for future deployment, as part of a research effort currently underway to instrument coastal homes in Florida to monitor roof wind pressures during hurricanes. The proposed wireless system will replace the current system that involves a large amount of hardwired connections from the sensors to the data processing unit that requires labor intensive wiring and preparation of the home. The paper describes comparison studies and field tests to assess the performance of the system. The new system offers the advantages of light hardware, ease of installation, capacity for 48 hours of continuous data acquisition, good frequency and amplitude responses, and a relatively simple maintenance. However, the tests also show that the shape of the shell that has been previously used to protect the sensors might interfere with the proper measurement of the pressures.

Enhancing Acute Kidney Injury Prediction through Integration of Drug Features in Intensive Care Units

  • Gabriel D. M. Manalu;Mulomba Mukendi Christian;Songhee You;Hyebong Choi
    • International journal of advanced smart convergence
    • /
    • v.12 no.4
    • /
    • pp.434-442
    • /
    • 2023
  • The relationship between acute kidney injury (AKI) prediction and nephrotoxic drugs, or drugs that adversely affect kidney function, is one that has yet to be explored in the critical care setting. One contributing factor to this gap in research is the limited investigation of drug modalities in the intensive care unit (ICU) context, due to the challenges of processing prescription data into the corresponding drug representations and a lack in the comprehensive understanding of these drug representations. This study addresses this gap by proposing a novel approach that leverages patient prescription data as a modality to improve existing models for AKI prediction. We base our research on Electronic Health Record (EHR) data, extracting the relevant patient prescription information and converting it into the selected drug representation for our research, the extended-connectivity fingerprint (ECFP). Furthermore, we adopt a unique multimodal approach, developing machine learning models and 1D Convolutional Neural Networks (CNN) applied to clinical drug representations, establishing a procedure which has not been used by any previous studies predicting AKI. The findings showcase a notable improvement in AKI prediction through the integration of drug embeddings and other patient cohort features. By using drug features represented as ECFP molecular fingerprints along with common cohort features such as demographics and lab test values, we achieved a considerable improvement in model performance for the AKI prediction task over the baseline model which does not include the drug representations as features, indicating that our distinct approach enhances existing baseline techniques and highlights the relevance of drug data in predicting AKI in the ICU setting.

Design of a Large-scale Task Dispatching & Processing System based on Hadoop (하둡 기반 대규모 작업 배치 및 처리 기술 설계)

  • Kim, Jik-Soo;Cao, Nguyen;Kim, Seoyoung;Hwang, Soonwook
    • Journal of KIISE
    • /
    • v.43 no.6
    • /
    • pp.613-620
    • /
    • 2016
  • This paper presents a MOHA(Many-Task Computing on Hadoop) framework which aims to effectively apply the Many-Task Computing(MTC) technologies originally developed for high-performance processing of many tasks, to the existing Big Data processing platform Hadoop. We present basic concepts, motivation, preliminary results of PoC based on distributed message queue, and future research directions of MOHA. MTC applications may have relatively low I/O requirements per task. However, a very large number of tasks should be efficiently processed with potentially heavy inter-communications based on files. Therefore, MTC applications can show another pattern of data-intensive workloads compared to existing Hadoop applications, typically based on relatively large data block sizes. Through an effective convergence of MTC and Big Data technologies, we can introduce a new MOHA framework which can support the large-scale scientific applications along with the Hadoop ecosystem, which is evolving into a multi-application platform.

Building an Annotated English-Vietnamese Parallel Corpus for Training Vietnamese-related NLPs

  • Dien Dinh;Kiem Hoang
    • Proceedings of the IEEK Conference
    • /
    • summer
    • /
    • pp.103-109
    • /
    • 2004
  • In NLP (Natural Language Processing) tasks, the highest difficulty which computers had to face with, is the built-in ambiguity of Natural Languages. To disambiguate it, formerly, they based on human-devised rules. Building such a complete rule-set is time-consuming and labor-intensive task whilst it doesn't cover all the cases. Besides, when the scale of system increases, it is very difficult to control that rule-set. So, recently, many NLP tasks have changed from rule-based approaches into corpus-based approaches with large annotated corpora. Corpus-based NLP tasks for such popular languages as English, French, etc. have been well studied with satisfactory achievements. In contrast, corpus-based NLP tasks for Vietnamese are at a deadlock due to absence of annotated training data. Furthermore, hand-annotation of even reasonably well-determined features such as part-of-speech (POS) tags has proved to be labor intensive and costly. In this paper, we present our building an annotated English-Vietnamese parallel aligned corpus named EVC to train for Vietnamese-related NLP tasks such as Word Segmentation, POS-tagger, Word Order transfer, Word Sense Disambiguation, English-to-Vietnamese Machine Translation, etc.

  • PDF