• 제목/요약/키워드: data locality

Search Result 237, Processing Time 0.038 seconds

A Case Study of a Navigator Optimization Process

  • Cho, Doosan
    • International journal of advanced smart convergence
    • /
    • v.6 no.1
    • /
    • pp.26-31
    • /
    • 2017
  • When mobile navigator device accesses data randomly, the cache memory performance is rapidly deteriorated due to low memory access locality. For instance, GPS (General Positioning System) of navigator program for automobiles or drones, that are currently in common use, uses data from 32 satellites and computes current position of a receiver. This computation of positioning is the major part of GPS which accounts more than 50% computation in the program. In this computation task, the satellite signals are received in real time and stored in buffer memories. At this task, since necessary data cannot be sequentially stored, the data is read and used at random. This data accessing patterns are generated randomly, thus, memory system performance is worse by low data locality. As a result, it is difficult to process data in real time due to low data localization. Improving the low memory access locality inherited on the algorithms of conventional communication applications requires a certain optimization technique to solve this problem. In this study, we try to do optimizations with data and memory to improve the locality problem. In experiment, we show that our case study can improve processing speed of core computation and improve our overall system performance by 14%.

An advanced reversible data hiding algorithm based on the similarity between neighboring pixels

  • Jung, Soo-Mok
    • Journal of the Korea Society of Computer and Information
    • /
    • v.21 no.2
    • /
    • pp.33-42
    • /
    • 2016
  • In this paper, an advanced reversible data hiding algorithm which takes the advantage of the spatial locality in image was proposed. Natural image has a spatial locality. The pixel value of a natural image is similar to the values of neighboring pixels. So, using the neighboring pixel values, it is possible to precisely predict the pixel value. Frequency increases significantly at the peak point of the difference histogram using the predicted values. Therefore, it is possible to increase the amount of data to be embedded. By using the proposed algorithm, visually high quality stego-image can be generated, the original cover image and the embedded data can be extracted from the stego-image without distortion. The embedding data into the cover image of the proposed algorithm is much lager than that of the previous algorithm. The performance of the proposed algorithm was verified by experiment. The proposed algorithm is very useful for the reversible data hiding.

Data Deduplication Method using Locality-based Chunking policy for SSD-based Server Storages (SSD 기반 서버급 스토리지를 위한 지역성 기반 청킹 정책을 이용한 데이터 중복 제거 기법)

  • Lee, Seung-Kyu;Kim, Ju-Kyeong;Kim, Deok-Hwan
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.50 no.2
    • /
    • pp.143-151
    • /
    • 2013
  • NAND flash-based SSDs (Solid State Drive) have advantages of fast input/output performance and low power consumption so that they could be widely used as storages on tablet, desktop PC, smart-phone, and server. But, SSD has the disadvantage of wear-leveling due to increase of the number of writes. In order to improve the lifespan of the SSD, a variety of data deduplication techniques have been introduced. General fixed-size splitting method allocates fixed size of chunk without considering locality of data so that it may execute unnecessary chunking and hash key generation, and variable-size splitting method occurs excessive operation since it compares data byte-by-byte for deduplication. This paper proposes adaptive chunking method based on application locality and file name locality of written data in SSD-based server storage. The proposed method split data into 4KB or 64KB chunks adaptively according to application locality and file name locality of duplicated data so that it can reduce the overhead of chunking and hash key generation and prevent duplicated data writing. The experimental results show that the proposed method can enhance write performance, reduce power consumption and operation time compared to existing variable-size splitting method and fixed size splitting method using 4KB.

Performance Enhancement of A Massive Scientific Data Visualization System on Virtual Reality Environment by Using Data Locality (Data Locality를 활용한 VR환경에서의 대용량 데이터 가시화 시스템의 성능 개선)

  • Lee, Se-Hoon;Kim, Min-Ah;Lee, Joong-Yeon;Hur, Young-Ju
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2012.11a
    • /
    • pp.284-287
    • /
    • 2012
  • GLOVE(GLObal Virtual reality visualization Environment for scientific simulation)는 컴퓨팅 자원의 성능 향상으로 데이터 양이 급속히 증가한 응용 과학과 전산 시뮬레이션 분야의 대용량 과학 데이터를 효율적으로 가시화하여 분석하기 위한 도구이다. GLOVE의 데이터 관리자인 GDM(GLOVE Data Manager)은 대용량 데이터의 분산 병렬 가시화를 위해 분산 공유 메모리를 제공하는 GA(Global Array)를 이용해 테라 바이트 단위의 데이터를 실시간으로 처리한다. 그러나 대용량 과학 데이터를 가시화 하는 과정에서 기존의 Data Locality를 고려하지 않은 데이터 접근 방식으로 인한 성능 저하를 확인했다. 본 논문은 기존 GLOVE에서 발견한 성능 저하 현상을 밝히고, 이에 대한 해결 방법을 제시한다.

Efficient Locality-Aware Traffic Distribution in Apache Storm (Apache Storm에서 지역성을 고려한 효율적인 트래픽 분배)

  • Son, Siwoon;Lee, Sanghun;Moon, Yang-Sae
    • KIISE Transactions on Computing Practices
    • /
    • v.23 no.12
    • /
    • pp.677-683
    • /
    • 2017
  • Apache Storm is a representative real-time distributed processing system, which is able to process data streams quickly over distributed servers. Storm currently provides several stream grouping methods to distribute data traffic to multiple servers. Among them, the shuffle grouping may cause a processing delay problem and the local-or-shuffle grouping used to solve the problem may cause the problem of concentrating the traffic on a specific node. In this paper, we propose the locality-aware grouping to solve the problems that may arise in the existing Storm grouping methods. Experimental results show that the proposed locality-aware grouping is considerably superior to the existing shuffle grouping and the local-or-shuffle grouping. These results show that the new grouping is an excellent approach considering both the locality and load balancing which are limitations of the existing Storm.

Dual Cache Architecture for Low Cost and High Performance

  • Lee, Jung-Hoon;Park, Gi-Ho;Kim, Shin-Dug
    • ETRI Journal
    • /
    • v.25 no.5
    • /
    • pp.275-287
    • /
    • 2003
  • We present a high performance cache structure with a hardware prefetching mechanism that enhances exploitation of spatial and temporal locality. Temporal locality is exploited by selectively moving small blocks into the direct-mapped cache after monitoring their activity in the spatial buffer. Spatial locality is enhanced by intelligently prefetching a neighboring block when a spatial buffer hit occurs. We show that the prefetch operation is highly accurate: over 90% of all prefetches generated are for blocks that are subsequently accessed. Our results show that the system enables the cache size to be reduced by a factor of four to eight relative to a conventional direct-mapped cache while maintaining similar performance.

  • PDF

Comparative Analysis of Centralized Vs. Distributed Locality-based Repository over IoT-Enabled Big Data in Smart Grid Environment

  • Siddiqui, Isma Farah;Abbas, Asad;Lee, Scott Uk-Jin
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2017.01a
    • /
    • pp.75-78
    • /
    • 2017
  • This paper compares operational and network analysis of centralized and distributed repository for big data solutions in the IoT enabled Smart Grid environment. The comparative analysis clearly depicts that centralize repository consumes less memory consumption while distributed locality-based repository reduce network complexity issues than centralize repository in state-of-the-art Big Data Solution.

  • PDF

A Hot-Data Replication Scheme Based on Data Access Patterns for Enhancing Processing Speed of MapReduce (맵-리듀스의 처리 속도 향상을 위한 데이터 접근 패턴에 따른 핫-데이터 복제 기법)

  • Son, Ingook;Ryu, Eunkyung;Park, Junho;Bok, Kyoungsoo;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.13 no.11
    • /
    • pp.21-27
    • /
    • 2013
  • In recently years, with the growth of social media and the development of mobile devices, the data have been significantly increased. Hadoop has been widely utilized as a typical distributed storage and processing framework. The tasks in Mapreduce based on the Hadoop distributed file system are allocated to the map as close as possible by considering the data locality. However, there are data being requested frequently according to the data analysis tasks of Mapreduce. In this paper, we propose a hot-data replication mechanism to improve the processing speed of Mapreduce according to data access patterns. The proposed scheme reduces the task processing time and improves the data locality using the replica optimization algorithm on the high access frequency of hot data. It is shown through performance evaluation that the proposed scheme outperforms the existing scheme in terms of the load of access frequency.

A Block Structured Multimedia Data Prefetching (블록 구조형 멀티미디어 데이터의 선인출)

  • Kim Suk-Ju;Lee Byung-Kwon;Kim Suk-Il
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.29 no.1A
    • /
    • pp.53-64
    • /
    • 2004
  • As to medium data which is involved in the form of streaming for a multimedia application, it characterizes that spatial locality occurs strongly but temporal locality appears even weaker. In this paper, with regard to dynamic prefetching, we suggest a method to make the most of memory reference regularities which typically innate by nature in the multimedia data with strong spatial locality but with weak temporal locality. Especially, the suggested method has a remarkable capability such that it can reduce prefetching errors substantially compared to existing prefetching methods for an application Program which divides an way into small sub-blocks and, plus executes in the unit of sub-block. We carried out experiments to test the suggested method using various MediaBench benchmarks. From the results, we have confirmed that the occurrences of prefetching error decrease effectively than those of existing linear prefetching methods.

Modeling of Data References with Temporal Locality and Popularity Bias (시간 지역성과 인기 편향성을 가진 데이터 참조의 모델링)

  • Hyokyung Bahn
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.23 no.6
    • /
    • pp.119-124
    • /
    • 2023
  • This paper proposes a new reference model that can represent data access with temporal locality and popularity bias. Among existing reference models, the LRU-stack model can express temporal locality, which is a characteristic that the more recently referenced data has, the higher the probability of being referenced again. However, it cannot take into account differences in popularity of the data. Conversely, the independent reference model can reflect the different popularity of data, but has the limitation of not being able to model changes in data reference trends over time. The reference model presented in this paper overcomes the limitations of these two models and has the feature of reflecting both the popularity bias of data and their changes over time. This paper also examines the relationship between the cache replacement algorithm and the reference model, and shows the optimality of the proposed model.