• Title/Summary/Keyword: Small Files

Search Result 120, Processing Time 0.025 seconds

Processing Method of Mass Small File Using Hadoop Platform (하둡 플랫폼을 이용한 대량의 스몰파일 처리방법)

  • Kim, Chang-Bok;Chung, Jae-Pil
    • Journal of Advanced Navigation Technology
    • /
    • v.18 no.4
    • /
    • pp.401-408
    • /
    • 2014
  • Hadoop is composed with MapReduce programming model for distributed processing and HDFS distributed file system. Hadoop is suitable framework for big data processing, but processing of mass small files have many problems. The processing of mass small file in hadoop have problems to created one mapper per one file, and it have problems to needed many memory for store of meta information of file. This paper have comparison evaluation processing method of mass small file with various method in hadoop platform. The processing of general compression format is inadequate because of processing by one mapper regardless of data size. The processing of sequence and hadoop archive file is removed memory problem of namenode by compress and combine of small file. Hadoop archive file is faster then sequence file about combine time of small file. The processing using CombineFileInputFormat class is needed not combine of small file, and it have similar speed big data processing method.

A Method to Express Audio Binary Files by Color QR Codes and Its Application (오디오 바이너리 파일을 컬러 QR코드로 표현하는 방법과 그 응용)

  • Lee, Choong Ho
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.19 no.2
    • /
    • pp.47-53
    • /
    • 2018
  • This paper proposes a method to express an MP3 audio file by a series of color QR codes which can be printed on the paper. Moreover, the method can compress the data considerably. Firstly, an MP3 file is divided into many small files which have maximum capacity of binary file of a QR code. Secondly, the multiple files are converted to multiple black-and-white QR codes. Lastly, every three QR codes are combined into color QR codes. When combining, each of three black-and-white QR codes are regarded as red, green, blue components respectively. In this method, the areas of a color QR code where two QR codes are overlapped are expressed by the colors Cyan, Magenta and Yellow. And the areas where three components are overlapped are expressed by white color. Contrarily, the areas that no components are overlapped are expressed by white color. Experimentation result shows that an MP3 file with 8.5MB the original MP3 files are compressed with the compression rate around 15.7. This method has the advantage that can be used in the environments that the internet access is impossible.

Distributed File Placement and Coverage Expansion Techniques for Network Throughput Enhancement in Small-cell Network (소형셀 네트워크 전송용량 향상을 위한 분산 파일저장 및 커버리지 확장 기법)

  • Hong, Jun-Pyo
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.22 no.1
    • /
    • pp.183-189
    • /
    • 2018
  • This paper proposes distributed file placement and coverage expansion techniques for mitigating the traffic bottleneck in backhaul for small-cell networks. In order to minimize the backhaul load with limited memory space, the proposed scheme controls the coverage and file placement of base station according to file popularity distribution and memory space of base stations. In other words, since the cache hit ratio is low when there is small memory capacity or widespread file popularity distribution, the base stations expand its coverage and cache different set of files for the user located in overlapped area to exploit multiple cached file sets of base stations. Our simulation results show that the proposed scheme outperforms the conventional cache strategy in terms of network throughput when there is small memory capacity or widespread file popularity distribution.

Design and Implementation of File System Using Local Buffer Cache for Digital Convergence Devices (디지털 컨버전스 기기를 위한 지역 버퍼 캐쉬 파일 시스템 설계 및 구현)

  • Jeong, Geun-Jae;Cho, Moon-Haeng;Lee, Cheol-Hoon
    • The Journal of the Korea Contents Association
    • /
    • v.7 no.8
    • /
    • pp.21-30
    • /
    • 2007
  • Due to the growth of embedded systems and the development of semi-conductor and storage devices, digital convergence devises is ever growing. Digital convergence devices are equipments into which various functions such as communication, playing movies and wave files and electronic dictionarys are integrated. Example are portable multimedia players(PMPs), personal digital assistants(PDAs), and smart phones. Therefore, these devices need an efficient file system which manages and controls various types of files. In designing such file systems, the size constraint for small embedded systems as well as performance and compatibility should be taken into account. In this paper, we suggest the partial buffer cache technique. Contrary to the traditional buffer cache, the partial buffer cache is used for only the FAT meta data and write-only data. Simulation results show that we could enhance the write performance more than 30% when the file size is larger than about 100 KBytes.

The comparison of accuracy on three-unit fixed dental prosthesis made with CAD/CAM milling machines (치과 캐드캠 밀링장비에 따른 3본브릿지의 정확도 비교)

  • Bae, So-Yeon;Park, Jin-Young;Kim, Ji-Hwan;Kim, Hae-Young;Kim, Myung-Bae;Kim, Woong-Chul
    • Journal of Technologic Dentistry
    • /
    • v.37 no.1
    • /
    • pp.9-15
    • /
    • 2015
  • Purpose: The purpose of this study was to compare the accuracy of the maxillary three-unit fixed dental prosthesis (FDPs) made using two CAD/CAM milling machines : DCM Group(Dentaim CAD/CAM milling machine), WCM Group(Wieland CAD/CAM milling machine). Methods: Each of 10 duplicate models was scanned by blue light scanner(Identica, Medit, Korea), and the three-unit FDPs (STL file) was designed using DelcamCAD. A total of 20 three-unit FDPs was fabricated, comprising 2 groups of 10 specimens each (shrinkage ratio is 1:1). The first three-unit FDPs STL file was used as a CAD reference model (CRM). Obtained STL files by scanning the inner surface of three-unit FDPs were convened into the point clouds-ASC II files. Discrepancies between the point clouds and CRM were measured by superimposition software. Statistical methods to analyze the data were used non-parametric method. The mean (SD) values were compared by a Mann-Whitney U-test. Type one error rate was set at 0.05. Results: WCM group had small discrepancies with $2.17{\mu}m$ of mean value compared to $4.44{\mu}m$ in DCM group. The accuracy values between the two groups showed a sratistically significant difference (Table 2, p<.05). Conclusion: The accuracy of the three-unit fixed dental prosthesis(FDPs) made of two CAD/CAM milling machines were statistically different. Accuracy with which the prosthesis made of WCM group was superior.

Secure Deletion for Flash Memory File System (플래시메모리 파일시스템을 위한 안전한 파일 삭제 기법)

  • Sun, Kyoung-Moon;Choi, Jong-Moo;Lee, Dong-Hee;Noh, Sam-H.
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.13 no.6
    • /
    • pp.422-426
    • /
    • 2007
  • Personal mobile devices equipped with non-volatile storage such as MP3 player, PMP, cellular phone, and USB memory require safety for the stored data on the devices. One of the safety requirements is secure deletion, which is removing stored data completely so that the data can not be restored illegally. In this paper, we study how to design the secure deletion on Flash memory, commonly used as storage media for mobile devices. We consider two possible secure deletion policy, named zero-overwrite and garbage-collection respectively, and analyze how each policy affects the performance of Flash memory file systems. Then, we propose an adaptive file deletion scheme that exploits the merits of the two possible policies. Specifically, the proposed scheme applies the zero-overwrite policy for small files, whereas it employs the garbage-collection policy for large files. Real implementation experiments show that the scheme is not only secure but also efficient.

SBR-k(Sized-base replacement-k) : File Replacement in Data Grid Environments (SBR-k(Sized-based replacement-k) : 데이터 그리드 환경에서 파일 교체)

  • Park, Hong-Jin
    • The Journal of the Korea Contents Association
    • /
    • v.8 no.11
    • /
    • pp.57-64
    • /
    • 2008
  • The data grid computing provides geographically distributed storage resources to solve computational problems with large-scale data. Unlike cache replacement policies in virtual memory or web-caching replacement, an optimal file replacement policy for data grids is the one of the important problems by the fact that file size is very large. The traditional file replacement policies such as LRU(Least Recently Used), LCB-K(Least Cost Beneficial based on K), EBR(Economic-based cache replacement), LVCT(Least Value-based on Caching Time) have the problem that they have to predict requests or need additional resources to file replacement. To solve theses problems, this paper propose SBR-k(Sized-based replacement-k) that replaces files based on file size. The proposed policy considers file size to reduce the number of files corresponding to a requested file rather than forecasting the uncertain future for replacement. The results of the simulation show that hit ratio was similar when the cache size was small, but the proposed policy was superior to traditional policies when the cache size was large.

Lack of any Prognostic Role of Insulin-Like Growth Factor-1 Receptor in Non-Small Cell Lung Cancer

  • Dilli, Utku Donem;Yildırim, Mustafa;Suren, Dinc;Alikanoglu, Arsenal;Kaya, Vildan;Goktas, Sevil;Yildiz, Mustafa;Sezer, Cem;Gunduz, Seyda
    • Asian Pacific Journal of Cancer Prevention
    • /
    • v.15 no.14
    • /
    • pp.5753-5757
    • /
    • 2014
  • Background: The purpose of this study is to determine whether the IGF1R expression has a prognostic role in non-small cell lung cancer. Materials and Methods: Forty-seven patients histopathologically diagnosed with small cell lung cancer upon bronchoscopic biopsy or resection materials were included in the study. IGF1R expression was examined via immunohistochemical methods. In samples, >10% staining were assessed as positive and ${\leq}10%$ as negative. Information about demographic datas and treatments was obtained by retrospective searches of patient files. Results: IGF1R expression was determined as positive in 38 (80.9%) and as negative in 9 (19.1%) patients. There was no significant relation between IGF1R expression and histological sub-type, local invasion, lymph node and metastasis status (p=0.842, p=0.437, 0.064, 0.447, respectively). There was also no correlation with IGF1R expression and survival (p=0.141). Conclusions: There are conflicting results between IGF1R and its prognostic effects in the various studies. It has been claimed in some studies it is not related to prognosis as in our study, and in some studies it has been claimed that it is a good prognostic factor whereas in some studies it has been claimed as being a factor for worse prognosis. We think that IGF1R expression in non-small cell lung carcinoma patients deserves further analysis, because of its potential prognostic and predictive roles.

Auditory Images of Japanese /p/ by Koreans (일본어 /p/의 청각인상 연구)

  • Lee, Jae-Kang
    • Speech Sciences
    • /
    • v.11 no.3
    • /
    • pp.83-93
    • /
    • 2004
  • The objectives of this study are to analyze Korean speakers' pronunciations of various Japanese /p/ patterns and to provide desirable pronunciation models. This is a part of an ongoing research that aims to propose a useful method of teaching Japanese pronunciation of /p/ to Koreans. The experimental data consist of /p/ phonemes in word initial, word medial, and 'yoon' positions. Yoon must be written in small size after a letter and it only makes a syllable with the preceding letter in Japanese. There were 22 different phoneme positions. They were pronounced by 48 Japanese majoring students (24 females and 24 males), who were in their twenties and were raised in Daejeon and vicinity. The individual pronunciations were collected and digitized into 528 files. The results show that Koreans pronounced the Japanese phoneme /p/ in a variety of ways, according to the auditory environments in which the phoneme was tested: as [ph] in word initial, [pp] or [ph] in word medial, and [ph] in 'yoon', unlike native speakers who pronounced Japanese /p/ as [ph] in word initial, [pp] in word medial and, and [pp] or [ph] in 'yoon'.

  • PDF

The Design and Implementation of a Reusable Viewer Component

  • Kim, Hong-Gab;Lim, Young-Jae;Kim, Kyung-Ok
    • Proceedings of the KSRS Conference
    • /
    • 2002.10a
    • /
    • pp.66-69
    • /
    • 2002
  • This article outlines the capabilities of a viewer component called GridViewer, and proves its reusability. GridViewer was designed for the construction of the image display part of GIS or remote sensing application software, and consequently it is particularly straightforward to closely couple GridViewer with access to very large images. Displaying is performed through pyramid structure, which enables to treat very large dataset up to several gigabytes in size under the limited capability of PC. GridViewer is free from responsibility to handle various formats of raster data files by taking grid coverage, which is designed by OGC to promote interoperability between implementations done by data vendors and software vendors providing analysis and grid processing implementations. GridViewer differs from other such viewer by allowing for clients to extend its function and capability by using small set of methods originally implemented in it. We show its reusability and expandability by applying it in developing application programs performing various functions not supported originally by the GridViewer COM component.

  • PDF