• Title/Summary/Keyword: in-memory file systems

Search Result 121, Processing Time 0.031 seconds

An Efficient Cache Management Scheme for Load Balancing in Distributed Environments with Different Memory Sizes (상이한 메모리 크기를 가지는 분산 환경에서 부하 분산을 위한 캐시 관리 기법)

  • Choi, Kitae;Yoon, Sangwon;Park, Jaeyeol;Lim, Jongtae;Lee, Seokhee;Bok, Kyoungsoo;Yoo, Jaesoo
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.8
    • /
    • pp.543-548
    • /
    • 2015
  • Recently, volume of data has been growing dramatically along with the growth of social media and digital devices. However, the existing disk-based distributed file systems have limits to their performance of data processing or data access, due to I/O processing costs and bottlenecks. To solve this problem, the caching technique is being used to manage data in the memory. In this paper, we propose a cache management scheme to handle load balancing in a distributed memory environment. The proposed scheme distributes the data according to the memory size, n distributed environments with different memory sizes. If overloaded nodes occur, it redistributes the the access time of the caching data. In order to show the superiority of the proposed scheme, we compare it with an existing distributed cache management scheme through performance evaluation.

An Advanced Adaptive Garbage Collection Policy by Considering the Operation Characteristics (연산 특성을 고려한 향상된 적응적 가비지 컬렉션 정책)

  • Park, Song-Hwa;Lee, Jung-Hoon;Lee, Won-Oh;Kim, Hyun-Woo
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.13 no.5
    • /
    • pp.269-277
    • /
    • 2018
  • NAND flash memory has widely been used because of non-volatility, low power consumption and fast access time. However, it suffers from inability to provide update-in-place and the erase cycle is limited. The unit of read/write operation is a page and the unit of erase operation is a block. Moreover erase operation is slower than other operations. We proposed the Adaptive Garbage Collection (called "AGC") policy which focuses on not only reducing garbage collection process time for real-time guarantee but also wear-leveling for a flash memory lifetime. The AGC performs better than Cost-benefit policy and Greedy policy. But the AGC does not consider the operation characteristics. So we proposed the Advanced Adaptive Garbage Collection (called "A-AGC") policy which considers the page write operation count and block erase operation count. The A-AGC reduces the write operations by considering the data update frequency and update data size. Also, it reduces the erase operations by considering the file fragmentation. We implemented the A-AGC policy and measured the performance compared with the AGC policy. Simulation results show that the A-AGC policy performs better than AGC, specially for append operation.

Development of a Real-time Error-detection System;The Case study of an Electronic Jacquard

  • Huh, Jae-Yeong;Seo, Chang-Jun
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2003.10a
    • /
    • pp.2588-2593
    • /
    • 2003
  • Any system has the possibility of an error occurrence. Even if trivial errors were occurred, the original system would be fatally affected by the occurring errors. Accordingly, the error detection must be demanded. In this paper, we developed a real-time error detection system would be able to apply to an electronic Jacquard system. A Jacquard is a machine, which controls warps while weaving textiles, for manufacturing patterned cloth. There are two types of mechanical and electronic Jacquard. An electronic Jacquard is better than a mechanical Jacquard in view of the productivity and realizability for weaving various cloths. Recent weaving industry is growing up increasingly due to the electronic Jacquard. But, the problem of wrong weaving from error data exists in the electronic Jacquard. In this research, a real-time error detection system for an electronic Jacquard is developed for detecting errors in an electronic Jacquard in real-time. The real-time system is constructed using PC-based embedded system architecture. The system detects the occurring errors in real-time by storing 1344 data transferred in serial from an electronic Jacquard into memory, and then by comparing synchronously 1344 data stored into memory with 1344 data in a design file before the next data would be transferred to the Jacquard for weaving. The information of detected errors are monitored to the screen and stored into a file in real-time as the outputs of the system. In this research, we solve the problem of wrong weaving through checking the weaving data and detecting the occurred errors of an electronic Jacquard in real-time.

  • PDF

Flash-Conscious Storage Management Method for DBMS using Dynamic Log Page Allocation (동적 로그 페이지 할당을 이용한 플래시-고려 DBMS의 스토리지 관리 기법)

  • Song, Seok-Il;Khil, Ki-Jeong;Choi, Kil-Seong
    • Journal of Advanced Navigation Technology
    • /
    • v.14 no.5
    • /
    • pp.767-774
    • /
    • 2010
  • Due to advantages of NAND flash memory such as non-volatility, low access latency, low energy consumption, light weight, small size and shock resistance, it has become a better alternative over traditional magnetic disk drives, and has been widely used. Traditional DBMSs including mobile DBMSs may run on flash memory without any modification by using Flash Translation Layer (FTL), which emulates a random access block device to hide the characteristics of flash memory such as "erase-before-update". However, most existing FTLs are optimized for file systems, not for DBMSs, and traditional DBMSs are not aware of them. Also, traditional DBMSs do not consider the characteristics of flash memory. In this paper, we propose a flash-conscious storage system for DBMSs that utilizes flash memory as a main storage medium, and carefully put the characteristics of flash memory into considerations. The proposed flash-conscious storage system exploits log records to avoid costly update operations. It is shown that the proposed storage system outperforms the state.

An Evaluation of Multimedia Data Downstream with PDA in an Infrastructure Network

  • Hong, Youn-Sik;Hur, Hye-Sun
    • Journal of Information Processing Systems
    • /
    • v.2 no.2
    • /
    • pp.76-81
    • /
    • 2006
  • A PDA is used mainly for downloading data from a stationary server such as a desktop PC in an infrastructure network based on wireless LAN. Thus, the overall performance depends heavily on the performance of such downloading with PDA. Unfortunately, for a PDA the time taken to receive data from a PC is longer than the time taken to send it by 53%. Thus, we measured and analyzed all possible factors that could cause the receiving time of a PDA to be delayed with a test bed system. There are crucial factors: the TCP window size, file access time of a PDA, and the inter-packet delay that affects the receiving time of a PDA. The window size of a PDA during the downstream is reduced dramatically to 686 bytes from 32,581 bytes. In addition, because flash memory is embedded into a PDA, writing data into the flash memory takes twice as long as reading the data from it. To alleviate these, we propose three distinct remedies: First, in order to keep the window size at a sender constant, both the size of a socket send buffer for a desktop PC and the size of a socket receive buffer for a PDA should be increased. Second, to shorten its internal file access time, the size of an application buffer implemented in an application should be doubled. Finally, the inter-packet delay of a PDA and a desktop PC at the application layer should be adjusted asymmetrically to lower the traffic bottleneck between these heterogeneous terminals.

Study of In-Memory based Hybrid Big Data Processing Scheme for Improve the Big Data Processing Rate (빅데이터 처리율 향상을 위한 인-메모리 기반 하이브리드 빅데이터 처리 기법 연구)

  • Lee, Hyeopgeon;Kim, Young-Woon;Kim, Ki-Young
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.12 no.2
    • /
    • pp.127-134
    • /
    • 2019
  • With the advancement of IT technology, the amount of data generated has been growing exponentially every year. As an alternative to this, research on distributed systems and in-memory based big data processing schemes has been actively underway. The processing power of traditional big data processing schemes enables big data to be processed as fast as the number of nodes and memory capacity increases. However, the increase in the number of nodes inevitably raises the frequency of failures in a big data infrastructure environment, and infrastructure management points and infrastructure operating costs also increase accordingly. In addition, the increase in memory capacity raises infrastructure costs for a node configuration. Therefore, this paper proposes an in-memory-based hybrid big data processing scheme for improve the big data processing rate. The proposed scheme reduces the number of nodes compared to traditional big data processing schemes based on distributed systems by adding a combiner step to a distributed system processing scheme and applying an in-memory based processing technology at that step. It decreases the big data processing time by approximately 22%. In the future, realistic performance evaluation in a big data infrastructure environment consisting of more nodes will be required for practical verification of the proposed scheme.

Embedded File System for Ubiquitous Computing (유비쿼터스 컴퓨팅을 위한 임베디드 파일시스템)

  • Lee, Byung-Kwon;Ju, Young-Kwan;Kim, Suk-Il;Jeon, Joong-Nam
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.14 no.4
    • /
    • pp.424-430
    • /
    • 2004
  • This paper explains the construction of the filesystems which could be utilized in embedded systems as an implementation of ubiquitous computing. It includes the formal architecture of filesystem hierarchy for the DOC (Disk-On-Chip) filesystem and the flash filesystem based on the MTD (Memory Technology Devices). For DOC, the root filesystem and the user filesystem are constructed by the TrueFFS supported by the M-Systems. For MTD filesystem, the root filesystem is implemented in the fast RAM disk, and the user filesystem is implemented in the JFFS2 that supports large capacity. In order to support the GUI filesystem, the porting process of Qt/E is also included in this paper.

Design of Fast Operation Method In NAND Flash Memory File System (NAND 플래시 메모리 파일 시스템에 빠른 연산을 위한 설계)

  • Jin, Jong-Won;Lee, Tae-Hoon;Chung, Ki-Dong
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.14 no.1
    • /
    • pp.91-95
    • /
    • 2008
  • Flash memory is widely used in embedded systems because of its benefits such as non-volatile, shock resistant, and low power consumption. But NAND flash memory suffers from out-place-update, limited erase cycles, and page based read/write operations. To solve these problems, log-structured filesystem was proposed such as YAFFS. However, YAFFS sequentially retrieves an array of all block information to allocate free block for a write operation. Also before the write operation, YAFPS read the array of block information to find invalid block for erase. These could reduce the performance of the filesystem. This paper suggests fast operation method for NAND flash filesystem that solves the above-mentioned problems. We implemented the proposed methods in YAFFS. And we measured the performance compared with the original technique.

Implementation of a File System for Flash Memory (플래시 메모리를 위한 파일 시스템의 구현)

  • Park, Sang-Ho;Ahn, Woo-Hyun;Park, Dae-Yeon;Kim, Jeong-Ki;Park, Sung-Min
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.7 no.5
    • /
    • pp.402-415
    • /
    • 2001
  • Advantages of flash memories are their shock resistance and fast read speed, which is much faster than that of a HDD. Because of these characteristics, they are increasingly used in the traditional household electric appliance and portable handset and therefore, development of file systems which use them as storage medium is increasingly needed. But they have two problems as storage medium. First, data stored in them cannot be overwritten: it must be erased before new data can be stored. Unfortunately, this erase operation usually takes about one second. Consequently, updating data in flash memories takes long time. In this paper, their problem is solved by using a data update mechanism like LFS(Log-structured File System). Second, their erase operations are restricted. We propose novel cleaning policy in order to increase the life cycle. We implemented FAT file system, which is suitable to small storage medium and solved problems, which usually happen in implementing FAT. We evaluated the performance of sequential writes and random writes on our implemented flash file system.

  • PDF

Enhanced resource scheduling in Grid considering overload of different attributes

  • Hao, Yongsheng
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.3
    • /
    • pp.1071-1090
    • /
    • 2016
  • Most of scheduling methods in the Grid only consider one special attribute of the resource or one aspect of QoS (Quality of Service) of the job. In this paper, we focus on the problem that how to consider two aspects simultaneously. Based on the requirements of the jobs and the attributes of the resources, jobs are categorized into three kinds: CPU-overload, memory-overload, and bandwidth-overload jobs. One job may belong to different kinds according to different attributes. We schedule the jobs in different categories in different orders, and then propose a scheduling method-MTS (multiple attributes scheduling method) to schedule Grid resources. Based on the comparisons between our method, Min-min, ASJS (Adaptive Scoring Job Scheduling), and MRS (Multi-dimensional Scheduling) show: (1) MTS reduces the execution time more than 15% to other methods, (2) MTS improves the number of the finished jobs before the deadlines of the jobs, and (3) MTS enhances the file size of transmitted files (input files and output files) and improves the number of the instructions of the finished jobs.