• Title/Summary/Keyword: data scalability

Search Result 574, Processing Time 0.024 seconds

Design and Evaluation of a High-performance Journaling Scheme for Non-volatile Memory (비휘발성 메모리를 고려한 고성능 저널링 기법 설계 및 평가)

  • Han, Hyuck
    • The Journal of the Korea Contents Association
    • /
    • v.20 no.8
    • /
    • pp.368-374
    • /
    • 2020
  • Journaling file systems (JFS) manage changes of file systems not yet committed in a data structure known as a journal to restore the file system in the event of an unexpected failure. Extra write operations required for journaling negatively affect the performance of JFS. The high-performance and byte-addressable non-volatile memory (NVM) was expected to easily mitigate these performance problems by providing NVM space as journal storage. However, even with such non-volatile memory technologies, performance problems still arise due to scalability problems inherent in processing transactions of JFS. To solve this problem, we proposes a technique for processing file system transactions for scalable performance. To this end, lock-free data structures are used and multiple I/O requests are allowed to simultaneously be processed on high-performance storage devices with multiple I/O channels. We evaluate the file system with the proposed technique by comparing the original ext4 file system and the recent proposed NVM-based journaling file system on a multi-core server, and experimental results show that our file system has better performance (up-to 2.9/2.3 times) than the original ext4 file system and the recent NVM-based journaling file system, respectively.

Generator of Dynamic User Profiles Based on Web Usage Mining (웹 사용 정보 마이닝 기반의 동적 사용자 프로파일 생성)

  • An, Kye-Sun;Go, Se-Jin;Jiong, Jun;Rhee, Phill-Kyu
    • The KIPS Transactions:PartB
    • /
    • v.9B no.4
    • /
    • pp.389-390
    • /
    • 2002
  • It is important that acquire information about if customer has some habit in electronic commerce application of internet base that led in recommendation service for customer in dynamic web contents supply. Collaborative filtering that has been used as a standard approach to Web personalization can not get rapidly user's preference change due to static user profiles and has shortcomings such as reliance on user ratings, lack of scalability, and poor performance in the high-dimensional data. In order to overcome this drawbacks, Web usage mining has been prevalent. Web usage mining is a technique that discovers patterns from We usage data logged to server. Specially. a technique that discovers Web usage patterns and clusters patterns is used. However, the discovery of patterns using Afriori algorithm creates many useless patterns. In this paper, the enhanced method for the construction of dynamic user profiles using validated Web usage patterns is proposed. First, to discover patterns Apriori is used and in order to create clusters for user profiles, ARHP algorithm is chosen. Before creating clusters using discovered patterns, validation that removes useless patterns by Dempster-Shafer theory is performed. And user profiles are created dynamically based on current user sessions for Web personalization.

An Extended Virtual LAM System Deploying Multiple Route Server (다중 라우트 서버를 두는 확장된 가상랜 시스템)

  • Seo, Ju-Yeon;Lee, Mee-Jeong
    • Journal of KIISE:Information Networking
    • /
    • v.29 no.2
    • /
    • pp.117-128
    • /
    • 2002
  • Virtual LAN (VLAN) is an architecture to enable communication between end stations as if they were on the same LAN regardless of their physical locations. VLAN defines a limited broadcast domain to reduce the bandwidth waste. The Newbridge Inc. developed a layer 3 VLAN product called VIVID, which configures a VLAN based on W subnet addresses. In a VIVID system, a single route server is deployed for address resolution, VLAN configuration, and data broadcasting to a VLAN. If the size of the network, over which the VLANS supported by the VIVID system spans, becomes larger, this single route server could become a bottleneck point of the system performance. One possible approach to cope with this problem is to deploy multiple route servers. We propose two architectures, organic and independent, to expand the original VIVID system to deploy multiple route servers. A course of simulations are done to analyze the performance of each architecture that we propose. The simulation results show that the performances of the proposed architectures depend on the lengths of VLAN broadcasting sessions and the number of broadcast data frames generated by a session. It has also been shown that there are tradeoffs between the scalability of the architecture and their efficiency in data transmissions.

Multi-blockchain model ensures scalability and reliability based on intelligent Internet of Things (지능형 사물인터넷 기반의 확장성과 신뢰성을 보장하는 다중 블록체인 모델)

  • Jeong, Yoon-Su;Kim, Yong-Tae
    • Journal of Convergence for Information Technology
    • /
    • v.11 no.3
    • /
    • pp.140-146
    • /
    • 2021
  • As the environment using intelligent IoT devices increases, various studies are underway to ensure the integrity of information sent and received from intelligent IoT devices. However, all IoT information generated in heterogeneous environments is not fully provided with reliable protocols and services. In this paper, we propose an intelligent-based multi-blockchain model that can extract only critical information among various information processed by intelligent IoT devices. In the proposed model, blockchain is used to ensure the integrity of IoT information sent and received from IoT devices. The proposed model uses the correlation index of the collected information to trust a large number of IoT information to extract only the information with a high correlation index and bind it with blockchain. This is because the collected information can be extended to the n-tier structure as well as guaranteed reliability. Furthermore, since the proposed model can give weight information to the collection information based on blockchain, similar information can be selected (or bound) according to priority. The proposed model is able to extend the collection information to the n-layer structure while maintaining the data processing cost processed in real time regardless of the number of IoT devices.

Intelligent Transportation System (ITS) research optimized for autonomous driving using edge computing (엣지 컴퓨팅을 이용하여 자율주행에 최적화된 지능형 교통 시스템 연구(ITS))

  • Sunghyuck Hong
    • Advanced Industrial SCIence
    • /
    • v.3 no.1
    • /
    • pp.23-29
    • /
    • 2024
  • In this scholarly investigation, the focus is placed on the transformative potential of edge computing in enhancing Intelligent Transportation Systems (ITS) for the facilitation of autonomous driving. The intrinsic capability of edge computing to process voluminous datasets locally and in a real-time manner is identified as paramount in meeting the exigent requirements of autonomous vehicles, encompassing expedited decision-making processes and the bolstering of safety protocols. This inquiry delves into the synergy between edge computing and extant ITS infrastructures, elucidating the manner in which localized data processing can substantially diminish latency, thereby augmenting the responsiveness of autonomous vehicles. Further, the study scrutinizes the deployment of edge servers, an array of sensors, and Vehicle-to-Everything (V2X) communication technologies, positing these elements as constituents of a robust framework designed to support instantaneous traffic management, collision avoidance mechanisms, and the dynamic optimization of vehicular routes. Moreover, this research addresses the principal challenges encountered in the incorporation of edge computing within ITS, including issues related to security, the integration of data, and the scalability of systems. It proffers insights into viable solutions and delineates directions for future scholarly inquiry.

QoS improving method of Smart Grid Application using WMN based IEEE 802.11s (IEEE 802.11s기반 WMN을 사용한 Smart Grid Application의 QoS 성능향상 방안 연구)

  • Im, Eun Hye;Jung, Whoi Jin;Kim, Young Hyun;Kim, Byung Chul;Lee, Jae Yong
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.51 no.1
    • /
    • pp.11-23
    • /
    • 2014
  • Wireless Mesh Network(WMN) has drawn much attention due to easy deployment and good scalability. Recently, major power utilities have been focusing on R&D to apply WMN technology in Smart Grid Network. Smart Grid is an intelligent electrical power network that can maximize energy efficiency through bidirectional communication between utility providers and customers with ICT(Information Communication Technology). It is necessary to guarantee QoS of some important data in Smart Grid system such as real-time data delivery. In this paper, we suggest QoS enhancement method for WMN based Smart Grid system using IEEE 802.11s. We analyze Smart Grid Application characteristics and apply IEEE 802.11s WMN scheme for Smart Grid in domestic power communication system. Performance evaluation is progressed using NS-2 simulator implementing IEEE 802.11s. The simulation results show that the QoS enhancement scheme can guarantee stable bandwidth irrespective of traffic condition due to IEEE 802.11s reservation mechanism.

SSQUSAR : A Large-Scale Qualitative Spatial Reasoner Using Apache Spark SQL (SSQUSAR : Apache Spark SQL을 이용한 대용량 정성 공간 추론기)

  • Kim, Jonghoon;Kim, Incheol
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.6 no.2
    • /
    • pp.103-116
    • /
    • 2017
  • In this paper, we present the design and implementation of a large-scale qualitative spatial reasoner, which can derive new qualitative spatial knowledge representing both topological and directional relationships between two arbitrary spatial objects in efficient way using Aparch Spark SQL. Apache Spark SQL is well known as a distributed parallel programming environment which provides both efficient join operations and query processing functions over a variety of data in Hadoop cluster computer systems. In our spatial reasoner, the overall reasoning process is divided into 6 jobs such as knowledge encoding, inverse reasoning, equal reasoning, transitive reasoning, relation refining, knowledge decoding, and then the execution order over the reasoning jobs is determined in consideration of both logical causal relationships and computational efficiency. The knowledge encoding job reduces the size of knowledge base to reason over by transforming the input knowledge of XML/RDF form into one of more precise form. Repeat of the transitive reasoning job and the relation refining job usually consumes most of computational time and storage for the overall reasoning process. In order to improve the jobs, our reasoner finds out the minimal disjunctive relations for qualitative spatial reasoning, and then, based upon them, it not only reduces the composition table to be used for the transitive reasoning job, but also optimizes the relation refining job. Through experiments using a large-scale benchmarking spatial knowledge base, the proposed reasoner showed high performance and scalability.

Design and Implementation of a Backup System for Object based Storage Systems (객체기반 저장시스템을 위한 백업시스템 설계 및 구현)

  • Yun, Jong-Hyeon;Lee, Seok-Jae;Jang, Su-Min;Yoo, Jae-Soo;Kim, Hong-Yeon;Kim, Jun
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.35 no.1
    • /
    • pp.1-17
    • /
    • 2008
  • Recently, the object based storage devices systems(OSDs) have been actively researched. They are different from existing block based storage systems(BSDs) in terms of the storage unit. The storage unit of the OSDs is an object that includes the access methods, the attributes of data, the security information, and so on. The object has no size limit and no influence on the internal storage structures. Therefore, the OSDs improve the I/O throughput and the scalability. But the backup systems for the OSDs still use the existing backup techniques for the BSDs. As a result, they need much backup time and do not utilize the characteristics of the OSDs. In this paper, we design and implement a new object based backup system that utilizes the features of the OSDs. Our backup system significantly improves the backup time over existing backup systems because the raw objects are directly transferred to the backup devices in our system. It also restores the backup data much faster than the existing systems when system failures occur. In addition, it supports various types of backup and restore requests.

Scalable and Accurate Intrusion Detection using n-Gram Augmented Naive Bayes and Generalized k-Truncated Suffix Tree (N-그램 증강 나이브 베이스 알고리즘과 일반화된 k-절단 서픽스트리를 이용한 확장가능하고 정확한 침입 탐지 기법)

  • Kang, Dae-Ki;Hwang, Gi-Hyun
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.13 no.4
    • /
    • pp.805-812
    • /
    • 2009
  • In many intrusion detection applications, n-gram approach has been widely applied. However, n-gram approach has shown a few problems including unscalability and double counting of features. To address those problems, we applied n-gram augmented Naive Bayes with k-truncated suffix tree (k-TST) storage mechanism directly to classify intrusive sequences and compared performance with those of Naive Bayes and Support Vector Machines (SVM) with n-gram features by the experiments on host-based intrusion detection benchmark data sets. Experimental results on the University of New Mexico (UNM) benchmark data sets show that the n-gram augmented method, which solves the problem of independence violation that happens when n-gram features are directly applied to Naive Bayes (i.e. Naive Bayes with n-gram features), yields intrusion detectors with higher accuracy than those from Naive Bayes with n-gram features and shows comparable accuracy to those from SVM with n-gram features. For the scalable and efficient counting of n-gram features, we use k-truncated suffix tree mechanism for storing n-gram features. With the k-truncated suffix tree storage mechanism, we tested the performance of the classifiers up to 20-gram, which illustrates the scalability and accuracy of n-gram augmented Naive Bayes with k-truncated suffix tree storage mechanism.

An Analysis of Operational Efficiency and Productivity for deep-sea fishing vessels in the North Pacific Ocean (북태평양 조업선박의 운영 효율성 및 생산성 분석)

  • Cho, Wooyoun;Jo, Geonsik;Yeo, Gitae
    • Journal of Korea Port Economic Association
    • /
    • v.30 no.2
    • /
    • pp.113-132
    • /
    • 2014
  • With the global warming phenomenon, the deep sea water area that fishing vessels can enter and operate is ever widening. For example, the Arctic Ocean recently has overall competitive advantages due to having many deep-sea fish stocks. The North Pacific region is a strategic coastal district, the closest access point of Arctic Ocean. For Korean fishing vessels which now operate in North Pacific region, and want to entry the Arctic Ocean, the analysis of technical efficiency is needed for preparing the better industry's future. This paper aims to analyze the relative efficiency, and select the low effective deep-sea fishing vessels in the North Pacific, and to suggest their desirables strategies. As a research methodology, Data Envelopment Analysis (DEA) and Malmquist Index are applied to 16 fishing vessels for the periods(2009 to 2013). To draw out the efficiency of targeted deep-sea fishing vessels, gross tons, horsepowers, and operating days are used as input variables while total catch stands for an output variable. As a result, CCR efficiency, BCC efficiency and scalability efficiency are measured to be 0.8405, 0.9484 and 0.8858 respectively for 5 years (2009 to 2013). In conclusion, 38% of total tons, 36% of horsepowers and 29% of operating days each fishing vessel should be reduced to keep their competitive powers.