• Title/Summary/Keyword: in-memory data management

Search Result 431, Processing Time 0.028 seconds

Efficient FTL Mapping Management for Multiple Sector Size-based Storage Systems with NAND Flash Memory (다중 섹터 사이즈를 지원하는 낸드 플래시 메모리 기반의 저장장치를 위한 효율적인 FTL 매핑 관리 기법)

  • Lim, Seung-Ho;Choi, Min
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.16 no.12
    • /
    • pp.1199-1203
    • /
    • 2010
  • Data transfer between host system and storage device is based on the data unit called sector, which can be varied depending on computer systems. If NAND flash memory is used as a storage device, the variant sector size can affect storage system performance since its operation is much related to sector size and page size. In this paper, we propose an efficient FTL mapping management scheme to support multiple sector size within one NAND flash memory based storage device, and analyze the performance effect and management overhead. According to the proposed scheme, the management overhead of proposed FTL management is lower than conventional scheme when various sector sizes are configured in computer systems, while performance is less degraded in comparison with single sector size support system.

A Study on Efficient Memory Management Using Machine Learning Algorithm

  • Park, Beom-Joo;Kang, Min-Soo;Lee, Minho;Jung, Yong Gyu
    • International journal of advanced smart convergence
    • /
    • v.6 no.1
    • /
    • pp.39-43
    • /
    • 2017
  • As the industry grows, the amount of data grows exponentially, and data analysis using these serves as a predictable solution. As data size increases and processing speed increases, it has begun to be applied to new fields by combining artificial intelligence technology as well as simple big data analysis. In this paper, we propose a method to quickly apply a machine learning based algorithm through efficient resource allocation. The proposed algorithm allocates memory for each attribute. Learning Distinct of Attribute and allocating the right memory. In order to compare the performance of the proposed algorithm, we compared it with the existing K-means algorithm. As a result of measuring the execution time, the speed was improved.

The Conceptual Design of Mass Memory Unit for High Speed Data Processing in the STSAT-3 (고속 데이터 처리를 위한 과학기술위성 3호 대용량 메모리 유닛의 개념 설계)

  • Seo, In-Ho;Oh, Dae-Soo;Myung, Noh-Hoon
    • Journal of the Korean Society for Aeronautical & Space Sciences
    • /
    • v.38 no.4
    • /
    • pp.389-394
    • /
    • 2010
  • This paper describes the conceptual design of mass memory unit for high speed data processing and mass memory management in the STSAT-3 compared to that of STSAT-2. The FPGA directly controls the data receiving from two payloads with the maximum 100Mbps speed and 32Gb mass memory management to satisfy these requirements. We used SRAM-based FPGA from XILINX having fast operating speed and large logic cells. Therefore, the Triple Modular Redundancy(TMR) and configuration memory scrubbing techniques will also be used to protect FPGA from Single Event Upset(SEU) in space.

An Approach for Stock Price Forecast using Long Short Term Memory

  • K.A.Surya Rajeswar;Pon Ramalingam;Sudalaimuthu.T
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.4
    • /
    • pp.166-171
    • /
    • 2023
  • The Stock price analysis is an increasing concern in a financial time series. The purpose of the study is to analyze the price parameters of date, high, low, and news feed about the stock exchange price. Long short term memory (LSTM) is a cutting-edge technology used for predicting the data based on time series. LSTM performs well in executing large sequence of data. This paper presents the Long Short Term Memory Model has used to analyze the stock price ranges of 10 days and 20 days by exponential moving average. The proposed approach gives better performance using technical indicators of stock price with an accuracy of 82.6% and cross entropy of 71%.

Dynamic Data Migration in Hybrid Main Memories for In-Memory Big Data Storage

  • Mai, Hai Thanh;Park, Kyoung Hyun;Lee, Hun Soon;Kim, Chang Soo;Lee, Miyoung;Hur, Sung Jin
    • ETRI Journal
    • /
    • v.36 no.6
    • /
    • pp.988-998
    • /
    • 2014
  • For memory-based big data storage, using hybrid memories consisting of both dynamic random-access memory (DRAM) and non-volatile random-access memories (NVRAMs) is a promising approach. DRAM supports low access time but consumes much energy, whereas NVRAMs have high access time but do not need energy to retain data. In this paper, we propose a new data migration method that can dynamically move data pages into the most appropriate memories to exploit their strengths and alleviate their weaknesses. We predict the access frequency values of the data pages and then measure comprehensively the gains and costs of each placement choice based on these predicted values. Next, we compute the potential benefits of all choices for each candidate page to make page migration decisions. Extensive experiments show that our method improves over the existing ones the access response time by as much as a factor of four, with similar rates of energy consumption.

A Design and Implementation on Large Data File Management Using Buffer Cache and Virtual Memory File (버퍼 캐쉬와 가상메모리 파일을 이용한 대형 데이터화일의 처리방법 설계 및 구현)

  • 김병철;신병석;조동섭;황희영
    • The Transactions of the Korean Institute of Electrical Engineers
    • /
    • v.41 no.7
    • /
    • pp.784-792
    • /
    • 1992
  • In this paper we design and implement a method for application programs to allow handling of large data files in DOS environment. In this method we use extended memory and hard disk as a data buffer. And we use a part of the conventional DOS memory as a buffer cache which allows the application program to use extended memory and hard disks transparently. Using buffer cache also allows us some speed improvement for the application program.

  • PDF

Mass Memory Operation for Telemetry Processing of LEO Satellite (저궤도위성 원격측정 데이터 처리를 위한 대용량 메모리 운용)

  • Chae, Dong-Seok;Yang, Seung-Eun;Cheon, Yee-Jin
    • Aerospace Engineering and Technology
    • /
    • v.11 no.2
    • /
    • pp.73-79
    • /
    • 2012
  • Because the contact time between satellite and ground station is very limited in LEO (Low Earth Orbit) satellite, all telemetry data generated on spacecraft bus are stored in a mass memory and downlinked to the ground together with real time data during the contact time. The mass memory is initialized in the first system initialization phase and the page status of each memory block is generated step by step. After the completion of the system initialization, the telemetry data are continuously stored and the stored data are played back to the ground by command. And the memory scrubbing is periodically performed for correction of single bit error which can be generated on harsh space environment. This paper introduces the mass memory operation method for telemetry processing of LEO satellite. It includes a general mass memory data structure, the methods of mass memory initialization, scrubbing, data storage and downlink, and mass memory management of primary and redundant mass memory.

Experimental Comparisons of Simplex Method Program's Speed with Various Memory Referencing Techniques and Data Structures (여러 가지 컴퓨터 메모리 참조 방법과 자료구조에 대한 단체법 프로그램 수행 속도의 비교)

  • Park, Chan-Kyoo;Lim, Sung-Mook;Kim, Woo-Jae;Park, Soon-Dal
    • IE interfaces
    • /
    • v.11 no.2
    • /
    • pp.149-157
    • /
    • 1998
  • In this paper, various techniques considering the characteristics of computer memory management are suggested, which can be used in the implementation of simplex method. First, reduction technique of indirect addressing, redundant references of memory, and scatter/gather technique are implemented, and the effectiveness of the techniques is shown. Loop-unrolling technique, which exploits the arithmetic operation mechanism of computer, is also implemented. Second, a subroutine frequently called is written in low-level language, and the effectiveness is proved by experimental results. Third, row-column linked list and Gustavson's data structure are compared as the data structure for the large sparse matrix in LU form. Last, buffering technique and memory-mapped file which can be used in reading large data file are implemented and the effectiveness is shown.

  • PDF

Development of Flash Memory Management Algorithm (플래쉬 메모리 관리 알고리즘 개발)

  • Park, In-Gyu
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.38 no.1
    • /
    • pp.26-45
    • /
    • 2001
  • The Flash memory market: is an exciting market that has quickly over the last 10 years. Recently Flash memory provides a high-density. truly non-volatile, high performance read write memory solutions, also is characterized by low power consumption, extreme ruggedness and high reliability. Flash memory is an optimum solution for large nonvolitilc storage operations such as solid file storage, digital video recorder, digital still camera, The MP3 player and other portable multimedia communication applications requiring non-volatility. Regardless of the type of Flash memory, Flash media management software is always required to manage the larger Flash memory block partitions. This is true, since Flash memory cannot be erased on the byte level common to memory, but must be erased on a block granularity. The management of a Flash memory manager requires a keen understanding of a Flash technology and data management methods. Though Flash memory's write performance is relatively slow, the suggested algorithm offers a higher maximum write performance. Algorithms so far developed is not suitable for applications which is requiring more fast and frequent accesses. But, the proposed algorithm is focused on the justifiable operation even in the circumstance of fast and frequent accesses.

  • PDF

Implementation of High Speed Big Data Processing System using In Memory Data Grid in Semiconductor Process (반도체 공정에서 인 메모리 데이터 그리드를 이용한 고속의 빅데이터 처리 시스템 구현)

  • Park, Jong-Beom;Lee, Alex;Kim, Tony
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.15 no.5
    • /
    • pp.125-133
    • /
    • 2016
  • Data processing capacity and speed are rapidly increasing due to the development of hardware and software in recent time. As a result, data usage is geometrically increasing and the amount of data which computers have to process has already exceeded five-thousand transaction per second. That is, the importance of Big Data is due to its 'real-time' and this makes it possible to analyze all the data in order to obtain accurate data at right time under any circumstances. Moreover, there are many researches about this as construction of smart factory with the application of Big Data is expected to have reduction in development, production, and quality management cost. In this paper, system using In-Memory Data Grid for high speed processing is implemented in semiconductor process which numerous data occur and improved performance is proven with experiments. Implemented system is expected to be possible to apply on not only the semiconductor but also any fields using Big Data and further researches will be made for possible application on other fields.