• Journal of Advanced Navigation Technology
• /
• v.18 no.4
• /
• pp.401-408
• /
• 2014

Hadoop is composed with MapReduce programming model for distributed processing and HDFS distributed file system. Hadoop is suitable framework for big data processing, but processing of mass small files have many problems. The processing of mass small file in hadoop have problems to created one mapper per one file, and it have problems to needed many memory for store of meta information of file. This paper have comparison evaluation processing method of mass small file with various method in hadoop platform. The processing of general compression format is inadequate because of processing by one mapper regardless of data size. The processing of sequence and hadoop archive file is removed memory problem of namenode by compress and combine of small file. Hadoop archive file is faster then sequence file about combine time of small file. The processing using CombineFileInputFormat class is needed not combine of small file, and it have similar speed big data processing method.

• Journal of information and communication convergence engineering
• /
• v.10 no.4
• /
• pp.405-410
• /
• 2012

Several attempts have been made to add journaling capability to a traditional file allocation table (FAT) file system. However, they encountered issues such as excessive system load or instability of the journaling data itself. If journaling data is saved as a file format, it can be corrupted by a user application. However, if journaling data is saved in a fixed area such as a reserved area, the storage can be physically corrupted because of excessive system load. To solve this problem, a new method that dynamically allocates journaling data is introduced. In this method, the journaling data is not saved as a file format. Using a reserved area and reserved FAT status entry of the FAT file system specification, the journaling data can be dynamically allocated and cannot be accessed by user applications. The experimental results show that this method is more stable and scalable than other log-based FAT file systems. HFAT was tested with more than 12,000 power failures and was stable.

• Journal of Information Technology Applications and Management
• /
• v.10 no.4
• /
• pp.169-183
• /
• 2003

In multimedia systems, very large files are stored and played. Because those files are read-centric and have low reusability, legacy file systems that are optimized for the use of many small files, do not show good performance in the multimedia systems. We implemented a new file system to deal with those characteristics. The new file system shows the superior performances in read, write and mixed operation to the Linux's EXT2 or EXT3 file systems. The new file system was implemented on the Linux operating system, and it also can be easily ported to other operating systems.

• Journal of Digital Contents Society
• /
• v.19 no.3
• /
• pp.497-503
• /
• 2018

RHEL 7, the leader in the enterprise Linux market, has dramatically increased the maximum support specification, such as file system size, file size, etc., by changing the default file system from EXT to XFS. It's not just an increase in support specifications, it's working on daemons, and it's showing excellent performance on high-performance disks such as high-capacity disks and solid state drives. Changes in the file system mean changes in direct operating techniques, such as changing related commands, changing backup tools, and changing disk quota settings. The changes to the XFS file system are making a lot of changes to the operation of the Linux system, but we believe that the position of the Linux operating system in the server field will become stronger.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.22 no.3
• /
• pp.557-567
• /
• 2018

There are currently various file encryption solutions for encrypting and storing files on disk. However, the existing file encryption solutions handle encryption and decryption all at once by file or directory. In this paper, we propose a system call supporting partial encryption function of the file. The user sets the encryption information with the system call interface at a portion where encryption of the file data is desired. And then the user writes file data, the data is encrypted and stored. Also if the user sets decryption information and reads the file data, the necessary part is decrypted by applying the set information. For the proposed system call, It consists of inspection module, management module, encryption module, decryption module, and HMAC module as per required system call. And it was implemented on the Linux environment. Also the operation of implemented system call was verified on the development board, and the performance was analyzed by measuring performance speed.

• Korean Journal of Computational Design and Engineering
• /
• v.1 no.2
• /
• pp.87-96
• /
• 1996

IGES is a file format which has gained widespread use but has certain limitations such as limited information coverage and ambiguous definitions. In order to overcome the limitations of existing neutral file formats, STEP has been developed as a more comprehensive mechanism for product data exchange by ISO. This paper describes a file translation system between IGES and STEP. In this system, three EXPRESS schemata are defined for IGES, STEP and the translation relationship between IGES and STEP. Object codes are generated from the schemata and linked with file access libraries to IGES and STEP files. The translation was verified by visualization and reverse translation. The system developed in this study can easily applied to translate other file formats because the file structure and translation relationship are defined in EXPRESS - a high level information modeling language.

• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.18 no.33
• /
• pp.1-10
• /
• 1995

The allocation of files is essential to the efficiency and effectiveness of a distributed system that must meet geographically dispersed data processing demands. In this paper, we address an optimization model that generates optimal file migration policies in distributed database systems. The proposed model is a more generalized model that includes system's capacity constraints - computing sites' storage capacity and communication networks' capacity - which have not taken into consideration in previous researches. Using this model, we can establish initial file allocation, file reallocation and file migration polices that minimize a system operating cost under system's capacity constraints at an initial system design or reorganization point The proposed model not only can be adopted by small-sized systems but also provides a foundation for effective and simple heuristics for adaptive file migration in large systems.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2013.11a
• /
• pp.15-17
• /
• 2013

HDFS(Hadoop Distribute File System)는 대용량 파일 처리를 목적으로 설계 되었으며 현재 이상적인 분산 파일 시스템으로 각광 받고 있다. 이러한 HDFS는 기존 분산파일 시스템과 많은 유사성을 가지고 있으나, Fault Tolerance를 제공하고, 데이터 엑세스 패턴을 스트리밍 방식으로 지원하여 대용량 파일을 효율적으로 저장할 수 있다는 차별성을 가지고 있다. 하지만 실제 HDFS 데이터 집합에는 Small file이 차지하는 비중이 상당히 높으며, 이러한 다수의 Small file 은 데이터 처리에 있어 높은 비용을 초래할 뿐 아니라 Master Node 의 파일 처리 및 메모리 성능에 악영향을 미친다. 따라서 본 논문에서는 HDFS에서 Small file 이 미치는 영향을 분석하고 이러한 문제점을 해결 할 수 있는 로컬 인덱스 파일기반의 파일 병합 기법을 제안한다.

• 한국HCI학회:학술대회논문집
• /
• 2006.02a
• /
• pp.902-908
• /
• 2006

We propose a scheme for more efficient navigation in a hierarchical file system. In the proposed scheme, a program running in the background computes the degree of relationship between a current file and others, and builds a list of the most related files. The current relationship metric being used by the program is a linear combination of five parameters: the name, the directory path, the type, the created time, and the last accessed time of a file. A simulated annealing algorithm is used in order to determine the weighting factors of the parameters. A set of experiments were conducted in order to access the effectiveness of the proposed scheme.

• Journal of Digital Contents Society
• /
• v.16 no.2
• /
• pp.299-306
• /
• 2015

In this paper, we research to secure stability of data storing/searching on NoSQL implemented on Distributed File System. When implementing NoSQL on Distributed File System, we faced that random write on Distributed File System is almost impossible. To solve this problem, a concept of Intermediate-File was employed, and then it has been achieved that our system resist any failure circumstance. Additionally, since we discovered its performance cannot be as fast as general File System, by redefining the file block unit for our NoSQL system, we have prevented a slowdown in system performance. As a result, we are able to develop highly scalable NoSQL as Distributed File System, which fulfills basic conditions of transaction: Atomicity, Consistency, Isolation, and Performance.