• Title/Summary/Keyword: in-memory data management

Search Result 437, Processing Time 0.026 seconds

Predictive Memory Allocation over Skewed Streams

  • Yun, Hong-Won
    • Journal of information and communication convergence engineering
    • /
    • v.7 no.2
    • /
    • pp.199-202
    • /
    • 2009
  • Adaptive memory management is a serious issue in data stream management. Data stream differ from the traditional stored relational model in several aspect such as the stream arrives online, high volume in size, skewed data distributions. Data skew is a common property of massive data streams. We propose the predicted allocation strategy, which uses predictive processing to cope with time varying data skew. This processing includes memory usage estimation and indexing with timestamp. Our experimental study shows that the predictive strategy reduces both required memory space and latency time for skewed data over varying time.

A Memory Mapping Technique to Reduce Data Retrieval Cost in the Storage Consisting of Multi Memories (다중 메모리로 구성된 저장장치에서 데이터 탐색 비용을 줄이기 위한 메모리 매핑 기법)

  • Hyun-Seob Lee
    • Journal of Internet of Things and Convergence
    • /
    • v.9 no.1
    • /
    • pp.19-24
    • /
    • 2023
  • Recently, with the recent rapid development of memory technology, various types of memory are developed and are used to improve processing speed in data management systems. In particular, NAND flash memory is used as a main media for storing data in memory-based storage devices because it has a nonvolatile characteristic that it can maintain data even at the power off state. However, since the recently studied memory-based storage device consists of various types of memory such as MRAM and PRAM as well as NAND flash memory, research on memory management technology is needed to improve data processing performance and efficiency of media in a storage system composed of different types of memories. In this paper, we propose a memory mapping scheme thought technique for efficiently managing data in the storage device composed of various memories for data management. The proposed idea is a method of managing different memories using a single mapping table. This method can unify the address scheme of data and reduce the search cost of data stored in different memories for data tiering.

High Speed Kernel Data Collection method for Analysis of Memory Workload (메모리 워크로드 분석을 위한 고속 커널 데이터 수집 기법)

  • Yoon, Jun Young;Jung, Seung Wan;Park, Jong Woo;Kim, Jung-Joon;Seo, Dae-Wha
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.2 no.11
    • /
    • pp.461-470
    • /
    • 2013
  • This paper proposes high speed kernel data collection method for analysis of memory workload, using technique of direct access to process's memory management structure. The conventional analysis tools have a slower data collection speed and they are lack of scalability due to collection only formalized memory information. The proposed method collects kernel data much faster than the conventional methods using technique of direct collect to process's memory information, page table, page structure in the memory management structure, and it can collect data which user wanted. We collect memory management data of the running process, and analyze its memory workload.

Adaptive Memory Management Method based on Utilization Ratio to Process Continuous Query (연속질의의 처리를 위한 이용률 기반의 적응적 메모리 관리 기법)

  • Baek, Sung-Ha;Lee, Dong-Wook;Eo, Sang-Hun;Chung, Weon-Il;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.11 no.2
    • /
    • pp.79-88
    • /
    • 2009
  • The volume of memory to store real-time data stream is varied dynamically. Continuous queries processing the data stream must manage the storage volume dynamically. In previous research, according to current volume of data a general memory manager which allocates and releases memory by a page unit is researched.However, the method frequently executes page allocation and release to store data stream. Moreover, particularly delayed queries can monopolize many of pages because the method directly allocates pages when a query has not enough memory. Focusing on the problems in memory management systems, this research proposes a memory management method which reduces the frequency of allocation and release and uniformly distributes pages for queries. The method can reduce the frequency of allocation and release through allocation based on utilization ratio of pages in each query and prevent memory monopoly through memory allocation which considers query delay.

  • PDF

Design of Memory-Resident GIS Database Systems

  • Lee, J. H.;Nam, K.W.;Lee, S.H.;Park, J.H.
    • Proceedings of the KSRS Conference
    • /
    • 2003.11a
    • /
    • pp.499-501
    • /
    • 2003
  • As semiconductor memory becomes cheaper, the memory capacity of computer system is increasing. Therefore computer system has sufficient memory for a plentiful spatial data. With emerging spatial application required high performance, this paper presents a GIS database system in main memory. Memory residence can provide both functionality and performance for a database management system. This paper describes design of DBMS for storing, querying, managing and analyzing for spatial and non-spatial data in main-memory. This memory resident GIS DBMS supports SQL for spatial query, spatial data model, spatial index and interface for GIS tool or applications.

  • PDF

Concurrency Control Method to Provide Transactional Processing for Cloud Data Management System

  • Choi, Dojin;Song, Seokil
    • International Journal of Contents
    • /
    • v.12 no.1
    • /
    • pp.60-64
    • /
    • 2016
  • As new applications of cloud data management system (CDMS) such as online games, cooperation edit, social network, and so on, are increasing, transaction processing capabilities for CDMS are required. Several transaction processing methods for cloud data management system (CDMS) have been proposed. However, existing transaction processing methods have some problems. Some of them provide limited transaction processing capabilities. Some of them are hard to be integrated with existing CDMSs. In this paper, we proposed a new concurrency control method to support transaction processing capability for CDMS to solve these problems. The proposed method was designed and implemented based on Spark, an in-memory distributed processing framework. It uses RDD (Resilient Distributed Dataset) model to provide fault tolerant to data in the main memory. In our proposed method, database stored in CDMS is loaded to main memory managed by Spark. The loaded data set is then transformed to RDD. In addition, we proposed a multi-version concurrency control method through immutable characteristics of RDD. Finally, we performed experiments to show the feasibility of the proposed method.

An Efficient Cache Management Scheme for Load Balancing in Distributed Environments with Different Memory Sizes (상이한 메모리 크기를 가지는 분산 환경에서 부하 분산을 위한 캐시 관리 기법)

  • Choi, Kitae;Yoon, Sangwon;Park, Jaeyeol;Lim, Jongtae;Lee, Seokhee;Bok, Kyoungsoo;Yoo, Jaesoo
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.8
    • /
    • pp.543-548
    • /
    • 2015
  • Recently, volume of data has been growing dramatically along with the growth of social media and digital devices. However, the existing disk-based distributed file systems have limits to their performance of data processing or data access, due to I/O processing costs and bottlenecks. To solve this problem, the caching technique is being used to manage data in the memory. In this paper, we propose a cache management scheme to handle load balancing in a distributed memory environment. The proposed scheme distributes the data according to the memory size, n distributed environments with different memory sizes. If overloaded nodes occur, it redistributes the the access time of the caching data. In order to show the superiority of the proposed scheme, we compare it with an existing distributed cache management scheme through performance evaluation.

Energy-Efficient Last-Level Cache Management for PCM Memory Systems

  • Bahn, Hyokyung
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.14 no.1
    • /
    • pp.188-193
    • /
    • 2022
  • The energy efficiency of memory systems is an important task in designing future computer systems as memory capacity continues to increase to accommodate the growing big data. In this article, we present an energy-efficient last-level cache management policy for future mobile systems. The proposed policy makes use of low-power PCM (phase-change memory) as the main memory medium, and reduces the amount of data written to PCM, thereby saving memory energy consumptions. To do so, the policy keeps track of the modified cache lines within each cache block, and replaces the last-level cache block that incurs the smallest PCM writing upon cache replacement requests. Also, the policy considers the access bit of cache blocks along with the cache line modifications in order not to degrade the cache hit ratio. Simulation experiments using SPEC benchmarks show that the proposed policy reduces the power consumption of PCM memory by 22.7% on average without degrading performances.

A Review of Data Management Techniques for Scratchpad Memory (스크래치패드 메모리를 위한 데이터 관리 기법 리뷰)

  • DOOSAN CHO
    • The Journal of the Convergence on Culture Technology
    • /
    • v.9 no.1
    • /
    • pp.771-776
    • /
    • 2023
  • Scratchpad memory is a software-controlled on-chip memory designed and used to mitigate the disadvantages of existing cache memories. Existing cache memories have TAG-related hardware control logic, so users cannot directly control cache misses, and their sizes are large and energy consumption is relatively high. Scratchpad memory has advantages in terms of size and energy consumption because it eliminates such hardware overhead, but there is a burden on software to manage data. In this study, data management techniques of scratchpad memory were classified and examined, and ways to maximize the advantages were discussed.

A Study on the Multi-Tactical Data Link Data Management (다중 전술 데이터링크 데이터 관리에 대한 연구)

  • Hwang, Jung-Eun;Lee, Kang;Jung, Suk-Ho
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.15 no.3
    • /
    • pp.457-464
    • /
    • 2020
  • ROK Forces operate Link-11, Link-16 and Korean Tactical Data Link System with Link-K(JTDLS: Joint Tactical Data Link System). As the change from Link-11 to Link-22 approaches, in the JTDLS Completed Business will be added tactical data link processing such as Link-22. In this paper, we propose the data management structure for data from multiple tactical data links. Also, we simulate to find ways to process a lot of data quickly using Shared Memory, In-Memory DB, Self Development DB in DLP, and then we confirm the result.