• Title/Summary/Keyword: Distributed Processing

Search Result 2,316, Processing Time 0.031 seconds

Frequency-Code Domain Contention in Multi-antenna Multicarrier Wireless Networks

  • Lv, Shaohe;Zhang, Yiwei;Li, Wen;Lu, Yong;Dong, Xuan;Wang, Xiaodong;Zhou, Xingming
    • Journal of Communications and Networks
    • /
    • v.18 no.2
    • /
    • pp.218-226
    • /
    • 2016
  • Coordination among users is an inevitable but time-consuming operation in wireless networks. It severely limit the system performance when the data rate is high. We present FC-MAC, a novel MAC protocol that can complete a contention within one contention slot over a joint frequency-code domain. When a node takes part in the contention, it generates randomly a contention vector (CV), which is a binary sequence of length equal to the number of available orthogonal frequency division multiplexing (OFDM) subcarriers. In FC-MAC, different user is assigned with a distinct signature (i.e., PN sequence). A node sends the signature at specific subcarriers and uses the sequence of the ON/OFF states of all subcarriers to indicate the chosen CV. Meanwhile, every node uses the redundant antennas to detect the CVs of other nodes. The node with the minimum CV becomes the winner. The experimental results show that, the collision probability of FC-MAC is as low as 0.05% when the network has 100 nodes. In comparison with IEEE 802.11, contention time is reduced by 50-80% and the throughput gain is up to 200%.

Adaptive Priority Queue-driven Task Scheduling for Sensor Data Processing in IoT Environments (사물인터넷 환경에서 센서데이터의 처리를 위한 적응형 우선순위 큐 기반의 작업 스케줄링)

  • Lee, Mijin;Lee, Jong Sik;Han, Young Shin
    • Journal of Korea Multimedia Society
    • /
    • v.20 no.9
    • /
    • pp.1559-1566
    • /
    • 2017
  • Recently in the IoT(Internet of Things) environment, a data collection in real-time through device's sensor has increased with an emergence of various devices. Collected data from IoT environment shows a large scale, non-uniform generation cycle and atypical. For this reason, the distributed processing technique is required to analyze the IoT sensor data. However if you do not consider the optimal scheduling for data and the processor of IoT in a distributed processing environment complexity increase the amount in assigning a task, the user is difficult to guarantee the QoS(Quality of Service) for the sensor data. In this paper, we propose APQTA(Adaptive Priority Queue-driven Task Allocation method for sensor data processing) to efficiently process the sensor data generated by the IoT environment. APQTA is to separate the data into job and by applying the priority allocation scheduling based on the deadline to ensure that guarantee the QoS at the same time increasing the efficiency of the data processing.

An Efficient Design and Implementation of an MdbULPS in a Cloud-Computing Environment

  • Kim, Myoungjin;Cui, Yun;Lee, Hanku
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.9 no.8
    • /
    • pp.3182-3202
    • /
    • 2015
  • Flexibly expanding the storage capacity required to process a large amount of rapidly increasing unstructured log data is difficult in a conventional computing environment. In addition, implementing a log processing system providing features that categorize and analyze unstructured log data is extremely difficult. To overcome such limitations, we propose and design a MongoDB-based unstructured log processing system (MdbULPS) for collecting, categorizing, and analyzing log data generated from banks. The proposed system includes a Hadoop-based analysis module for reliable parallel-distributed processing of massive log data. Furthermore, because the Hadoop distributed file system (HDFS) stores data by generating replicas of collected log data in block units, the proposed system offers automatic system recovery against system failures and data loss. Finally, by establishing a distributed database using the NoSQL-based MongoDB, the proposed system provides methods of effectively processing unstructured log data. To evaluate the proposed system, we conducted three different performance tests on a local test bed including twelve nodes: comparing our system with a MySQL-based approach, comparing it with an Hbase-based approach, and changing the chunk size option. From the experiments, we found that our system showed better performance in processing unstructured log data.

Distributed Processing System Design and Implementation for Feature Extraction from Large-Scale Malicious Code (대용량 악성코드의 특징 추출 가속화를 위한 분산 처리 시스템 설계 및 구현)

  • Lee, Hyunjong;Euh, Seongyul;Hwang, Doosung
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.8 no.2
    • /
    • pp.35-40
    • /
    • 2019
  • Traditional Malware Detection is susceptible for detecting malware which is modified by polymorphism or obfuscation technology. By learning patterns that are embedded in malware code, machine learning algorithms can detect similar behaviors and replace the current detection methods. Data must collected continuously in order to learn malicious code patterns that change over time. However, the process of storing and processing a large amount of malware files is accompanied by high space and time complexity. In this paper, an HDFS-based distributed processing system is designed to reduce space complexity and accelerate feature extraction time. Using a distributed processing system, we extract two API features based on filtering basis, 2-gram feature and APICFG feature and the generalization performance of ensemble learning models is compared. In experiments, the time complexity of the feature extraction was improved about 3.75 times faster than the processing time of a single computer, and the space complexity was about 5 times more efficient. The 2-gram feature was the best when comparing the classification performance by feature, but the learning time was long due to high dimensionality.

Improvement on The Complexity of Distributed Depth First Search Protocol (분산깊이 우선 탐색 프로토콜의 복잡도 개선을 위한 연구)

  • Choe, Jong-Won
    • The Transactions of the Korea Information Processing Society
    • /
    • v.3 no.4
    • /
    • pp.926-937
    • /
    • 1996
  • A graph traversal technique is a certain pattern of visiting nodes of a graph. Many special traversal techniques have been applied to solve graph related problems. For example, the depth first search technique has been used for finding strongly onnected components of a directed graph or biconnected components of a general graph. The distributed protocol to implement his depth first search technique on the distributed network can be divided into a fixed topology problem where there is no topological change and a dynamic topology problem which has some topological changes. Therefore, in this paper, we present a more efficient distributed depth first search protocol with fixed topology and a resilient distributed depth first search protocol where there are topological changes for the distributed network. Also, we analysed the message and time complexity of the presented protocols and showed the improved results than the complexities of the other distributed depth first search protocols.

  • PDF

An Internet-based computing framework for the simulation of multi-scale response of structural systems

  • Chen, Hung-Ming;Lin, Yu-Chih
    • Structural Engineering and Mechanics
    • /
    • v.37 no.1
    • /
    • pp.17-37
    • /
    • 2011
  • This paper presents a new Internet-based computational framework for the realistic simulation of multi-scale response of structural systems. Two levels of parallel processing are involved in this frame work: multiple local distributed computing environments connected by the Internet to form a cluster-to-cluster distributed computing environment. To utilize such a computing environment for a realistic simulation, the simulation task of a structural system has been separated into a simulation of a simplified global model in association with several detailed component models using various scales. These related multi-scale simulation tasks are distributed amongst clusters and connected to form a multi-level hierarchy. The Internet is used to coordinate geographically distributed simulation tasks. This paper also presents the development of a software framework that can support the multi-level hierarchical simulation approach, in a cluster-to-cluster distributed computing environment. The architectural design of the program also allows the integration of several multi-scale models to be clients and servers under a single platform. Such integration can combine geographically distributed computing resources to produce realistic simulations of structural systems.

Research on the Sharing Strategy of Electronic Book Resources in Universities in the Internet Era

  • Guiya Gao
    • Journal of Information Processing Systems
    • /
    • v.19 no.5
    • /
    • pp.590-601
    • /
    • 2023
  • University books are an important information resource. University book resources can be shared not only in the traditional paper form, but also electronic form under the background of the Internet. In order to better manage the sharing of electronic book resources in universities, this study put forward three resource sharing strategies: centralized sharing strategy, distributed sharing strategy, and centralized-distributed sharing strategy by analyzing the combined development of books and the Internet as well as the significance and development of book resource sharing. The centralized sharing strategy, however simple, was difficult to handle large traffic; while the resource nodes were independent and self-consistent, the distributed sharing strategy was not easy to find and had a high repetition rate. Combining the advantages of both strategies, the centralized-distributed sharing strategy was more suitable for the heterogeneous form of university book sharing. Finally, a teaching resources sharing platform for university libraries was designed based on the strategy of centralized and distributed sharing, and three interfaces including platform login, resource search, and resource release were displayed. The results of the simulated comparison experiment showed that centralized and distributed sharing strategies had limitations in resource searching and had low efficiencies; the efficiency of the centralized strategy reduced with an increase in search subjects; however, the centralized-distributed sharing strategy was able to search more resources efficiently and main stability.

Concurrency Control Method to Provide Transactional Processing for Cloud Data Management System

  • Choi, Dojin;Song, Seokil
    • International Journal of Contents
    • /
    • v.12 no.1
    • /
    • pp.60-64
    • /
    • 2016
  • As new applications of cloud data management system (CDMS) such as online games, cooperation edit, social network, and so on, are increasing, transaction processing capabilities for CDMS are required. Several transaction processing methods for cloud data management system (CDMS) have been proposed. However, existing transaction processing methods have some problems. Some of them provide limited transaction processing capabilities. Some of them are hard to be integrated with existing CDMSs. In this paper, we proposed a new concurrency control method to support transaction processing capability for CDMS to solve these problems. The proposed method was designed and implemented based on Spark, an in-memory distributed processing framework. It uses RDD (Resilient Distributed Dataset) model to provide fault tolerant to data in the main memory. In our proposed method, database stored in CDMS is loaded to main memory managed by Spark. The loaded data set is then transformed to RDD. In addition, we proposed a multi-version concurrency control method through immutable characteristics of RDD. Finally, we performed experiments to show the feasibility of the proposed method.

Rhipe Platform for Big Data Processing and Analysis (빅데이터 처리 및 분석을 위한 Rhipe 플랫폼)

  • Jung, Byung Ho;Shin, Ji Eun;Lim, Dong Hoon
    • The Korean Journal of Applied Statistics
    • /
    • v.27 no.7
    • /
    • pp.1171-1185
    • /
    • 2014
  • Rhipe that integrates R and Hadoop environment, made it possible to process and analyze massive amounts of data using a distributed processing environment. In this paper, we implemented multiple regression analysis using Rhipe with various data sizes of actual data and simulated data. Experimental results for comparing the computing speeds of pseudo-distributed and fully-distributed modes for configuring Hadoop cluster, showed fully-distributed mode was more fast than pseudo-distributed mode and computing speeds of fully-distributed mode were faster as the number of data nodes increases. We also compared the performance of our Rhipe with stats and biglm packages available on bigmemory. The results showed that our Rhipe was more fast than other packages owing to paralleling processing with increasing the number of map tasks as the size of data increases.

The Analysis of Mobile Agent System for the Wide Distributed Processing (광역 분산 처리 환경에서의 모바일 에이전트 시스템 분석)

  • 방정원
    • Journal of the Korea Society of Computer and Information
    • /
    • v.8 no.4
    • /
    • pp.132-137
    • /
    • 2003
  • Due to the advances of the Internet technologies and web-based applications, the demand for wide distributed processing is increased. Mobile Agents have been proposed as a model to meet with the requirement for such environment. But current mobile agent system has the technical restriction. The requirement of mobile agent for the wide distributed processing is analysed in mobile agent system architecture, programming paradigm and technologies.

  • PDF