• The Journal of the Korea Contents Association
• /
• v.14 no.11
• /
• pp.28-38
• /
• 2014

In this paper, we propose the distributed cache management scheme to efficiently access small files in Hadoop Distributed File Systems(HDFS). The proposed scheme can reduce the number of metadata managed by a name node since many small files are merged and stored in a chunk. It is also possible to reduce the file access costs, by keeping the information of requested files using the client cache and data node caches. The client cache keeps small files that a user requests and metadata. Each data node cache keeps the small files that are frequently requested by users. It is shown through performance evaluation that the proposed scheme significantly reduces the processing time over the existing scheme.

• Journal of the Korea Society of Computer and Information
• /
• v.20 no.8
• /
• pp.1-6
• /
• 2015

The MapReduce Program of Hadoop Distributed File System operates on any unspecified nodes due to distributed-parallel process and block replicate for data stability. Since it is difficult to guarantee the cache locality when a Solid State Drive is used as a cache in hadoop, cache hit-rate is decreased. In this paper, we suggest a method to improve cache hit rate by pre-loading the input data of the MapReduce onto the SSD cache. To perform this method, we estimated the blocks that are used on each node by using capacity scheduler and block metadata. Eventually we could increase the performance of SSD cache by loading the blocks onto SSD cache before the Map Task run.

• JSTS:Journal of Semiconductor Technology and Science
• /
• v.17 no.1
• /
• pp.15-22
• /
• 2017

Although there are many efficient techniques to minimize the speed gap between processor and the memory, it remains a bottleneck for various commercial implementations. Since secondary memory technologies are much slower than main memory, it is challenging to match memory speed to the processor. Usually, hard disk drives include semiconductor caches to improve their performance. A hit in the disk cache eliminates the mechanical seek time and rotational latency. To further improve performance a divided disk cache, subdivided between metadata and data, has been proposed previously. We propose a new algorithm to apply the SSD that is flash memory-based solid state drive by applying FTL. First, this paper evaluates the performance of such a disk cache via simulations using DiskSim. Then, we perform an experiment to evaluate the performance of the proposed algorithm.

• Journal of the Korea Society for Simulation
• /
• v.16 no.2
• /
• pp.9-15
• /
• 2007

As the Internet grows and the mass multimedia data become popular, the storage system migrates from DAS, where the storage and the server are directly connected, to SAN and NAS. SAN connects the storages with a separate network, and NAS provides only file services, connects the storages with IP network. However, SAN and NAS can not fulfill the needs for companies if used separately, thus are asked to be integrated. In this research, we propose an efficient data sharing method which employees the concept of GMC, Croup Master Cache for the integrated environment of SAN and NAS. GMC is based on MCI, Metadata server and Cluster system Integration, but tries to solve the high expansion cost problem with MCI. We introduce the basic concept of GMC, compare the performance of GMC with that of MCI using computer simulation.

• Proceedings of the Korean Information Science Society Conference
• /
• 1998.10a
• /
• pp.74-76
• /
• 1998

본 논문에서는 유닉스 파일 시스템에서의 버퍼캐쉬 크기에 따라 사용자 데이터와 메타데이터의 버퍼 캐쉬 히트율을 분석하였다. 그리고 메타 데이터가 유닉스 운영체제 파일 시스템의 성능에 미치는 영향을 분석하고 이를 기반으로 버퍼 캐쉬의 동적 특성과 성능의 장애 요인들을 분석하였다. 유닉스 운영체제에서 사용되는 사용자 데이터와 메타데이터에 대한 버퍼 캐쉬의 동적인 동작을 분석하기 위하여 trace-driven방법을 이용하였으며 이를 위하여 시뮬레이터를 작성.사용하였다. 파일 시스템은 특정 유닉스 버전에 영향을 받지 않기 위해 USF[1]에 기초하였고, 작업부하(workload)로는 Sprite- trace 데이터 중 allspice 서버에서 추출한 데이터를 사용하였다.

• Journal of KIISE:Computer Systems and Theory
• /
• v.31 no.1_2
• /
• pp.125-134
• /
• 2004

Recently, an on-chip compressed cache system was presented to alleviate the processor-memory Performance gap by reducing on-chip cache miss rate and expanding memory bandwidth. This research Presents an extended on-chip compressed cache system which also significantly expands main memory capacity. Several techniques are attempted to expand main memory capacity, on-chip cache capacity, and memory bandwidth as well as reduce decompression time and metadata size. To evaluate the performance of our proposed system over existing systems, we use execution-driven simulation method by modifying a superscalar microprocessor simulator. Our experimental methodology has higher accuracy than previous trace-driven simulation method. The simulation results show that our proposed system reduces execution time by 4-23% compared with conventional memory system without considering the benefits obtained from main memory expansion. The expansion rates of data and code areas of main memory are 57-120% and 27-36%, respectively.

• Proceedings of the Korea Contents Association Conference
• /
• 2014.11a
• /
• pp.47-48
• /
• 2014

최근 개인 사용자에게 최적화된 정보 및 서비스를 제공하기 위해 다수의 소형파일에 대한 분석이 요구되고 있다. 이런 요구사항을 고려해 본 논문에서는 통신주기에 따라 캐시 메타데이터 동기화를 적용한 분산 캐시 관리 기법을 제안한다. 제안하는 기법은 다수의 소형파일을 병합하여 청크에 저장함으로써, 네임노드가 관리하는 메타데이터의 양을 줄였으며, 클라이언트 캐시에 캐시 메타데이터를 추가적으로 저장하는 구조를 갖는다. 또한, 클라이언트와 네임노드 간 통신주기 조절을 통해 클라이언트의 캐시 메타데이터를 갱신함으로써 불필요한 처리시간을 감소시킨다.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2010.07a
• /
• pp.315-318
• /
• 2010

USN(Ubiquitous Sensor Network)기술이 다양한 분야에 적용하고 응용함으로서 센서 네트워크에서 발생하는 스트림 데이터를 효율적으로 처리하는 기술이 활발하게 연구되고 있다. 그리고 센서들에서 발생한 이기종 센서 스트림 데이터는 미들웨어의 메타데이터를 이용하여 센서 모니터링 환경의 상위 응용시스템에서 사용하기 적합한 센서 데이터로의 변환이 필요하다. 기존에 개발된 센서 데이터 처리 시스템에서는 동일한 노드나 지역에서 발생하는 센서 스트림 데이터에 대해서도 미들웨어에서의 불필요한 검색과 연산을 수행하기 때문에 본 논문에서는 캐쉬 테이블방식을 이용하여 중복적인 센서 스트림 데이터 처리의 수행을 줄일 수 있는 센서 스트림 데이터 처리 시스템의 설계를 제안한다.

• Journal of KIISE:Computing Practices and Letters
• /
• v.10 no.3
• /
• pp.282-291
• /
• 2004

There has been many researches in TopicMap and RDF which are approach to handle data efficiently with metadata. However, No researches has been performed to service and implement except for presentation and description. In this paper, We suggest the caching mechanism to support an efficient access of knowledgemap and practical knowledgemap service with implementation of TopicMap system. First, We propose a method to navigate Knowledgemap efficiently that includes advantage of former methods. Then, To transmit TopicMap efficiently, We suggest caching mechanism for knowledgemap. This method is that user will be able to navigate knowledgemap efficiently in the viewpoint of human, not application. Therefor the mechanism doesn't cash topics by logical or physical locality but clustering by information and characteristic value of TopicMap. Lastly, we suggest replace mechanism by using graph structure of TopicMap for efficiency of transmission.

• Journal of KIISE:Computing Practices and Letters
• /
• v.14 no.5
• /
• pp.537-541
• /
• 2008

Log-Structured File System(LFS) is a disk based file system that is optimized for improving the write performance. LFS gathers dirty data in memory as long as possible, and flushes all dirty data sequentially at once. In a real system, however, maintaining dirty data in memory should be flushed into a disk to meet file system consistency issues even if more memory is still available. This synchronizations increase the cleaner overhead of LFS and make LFS to write down more metadata into a disk. In this paper, by adapting Non-volatile RAM(NV-RAM) we modifies LFS and virtual memory subsystem to guarantee that LFS could gather enough dirty data in the memory and reduce small disk writes. By doing so, we improves the performance of LFS by around 2.5 times than the original LFS.