• 제목/요약/키워드: Huge file system

검색결과 36건 처리시간 0.017초

GIS에서 대용량 파일을 위한 대용량 공유 디스크 파일시스템의 메타데이터 구조 (Metadata Structrues of Huge Shared Disk File System for Large Files in GIS)

  • 김경배;이용주;박춘서;신범주
    • Spatial Information Research
    • /
    • 제10권1호
    • /
    • pp.93-106
    • /
    • 2002
  • 기존의 과일시스템은 소형의 과일을 효과적으로 저장하고 관리하기 위해서 설계되었다. 따라서 기존의 유닉스나 리눅스와 같은 과일 시스템은 지리정보시스템에서 발생되는 대용량의 지리정보 데이터를 효과적으로 처리하는 것은 어렵다. 본 논문에서는 지리정보시스템에서 발생되는 기가에서 테라바이트의 대용량 데이터 파일을 저장하기 위한 효과적인 메타데이터 구조 및 관리 기법을 제안한다. 제안된 기법에서는 대용량 파일을 저장하기 위해 동적 다단계 기법을 사용하고 있으며, 대용량의 파일 시스템을 제공하기 위하여 동적 비트맵 기법을 사용한다 본 논문에서 제안된 기법은 SAN 환경에서의 대용량 공유 디스크 파일시스템인 SANtopia에서 구현되었다.

  • PDF

대용량 파일시스템을 위한 동적 비트맵 (Dynamic Bitmap for Huge File System)

  • 김경배;이용주;박춘서;신범주
    • 정보처리학회논문지A
    • /
    • 제9A권3호
    • /
    • pp.287-294
    • /
    • 2002
  • 본 논문에서는 동적 비트맵 할당 기법을 사용하는 새로운 파일 시스템 기법에 대하여 제안한다. 기존의 파일 시스템들이 슈퍼블록, 아이노드, 디렉토리 엔트리와 같은 메타데이터를 위해 고정된 비트맵 구조의 의존하는 반면, 제안된 기법은 파일 시스템의 특성에 따라 비트맵과 할당 영역을 할당한다. 제안된 기법은 기존의 파일 시스템에서 디스크의 이용율이 파일의 크기에 의존하는 문제를 해결하였다. 성능평가를 통해서 동적 비트맵을 이용한 제안된 파일 시스템이 기존의 기법에 비해 효율적으로 디스크를 사용함을 보였다.

대용량 이메일 서비스를 위한 분산 구조 기반의 SMTP 서버 (A Distributed Architecture Based SMTP Server for Large Email Service)

  • 김영종;곽후근;정규식
    • 정보처리학회논문지C
    • /
    • 제16C권5호
    • /
    • pp.597-604
    • /
    • 2009
  • SMTP(Simple Mail Transfer Protocol) 서버는 네트워크를 기반으로 사용자의 메일을 전달 해주거나 저장하기 위해 만들어졌다. SMTP 서버는 대규모 서비스를 운영하거나 사용자가 많은 경우 또는 많은 양의 메일을 처리해야 하는 경우에는 SMTP 서버들을 복수로 구성할 수 있어야 한다. SMTP 서버의 메일 저장 공간인 파일시스템을 분리함으로써 복수의 SMTP 서버로 구성하는 것이 가능하며, 이때 분리된 파일시스템을 각각의 SMTP 서버들이 공유하기 위해 NFS(Network File System)를 사용하게 된다. 그러나 NFS는 네트워크 기반의 파일시스템으로써 파일시스템이 가지는 특성을 모두 가지고 있기 때문에, SMTP 서버들이 메일 저장 공간 공유를 위해 사용할 경우 불필요한 작업들로 인해 오버헤드가 발생하게 된다. 본 논문에서는 소켓(Socket)을 통해 직접 작업하는 방식을 사용하여 NFS를 사용함으로 인해 발생하는 작업 오버헤드를 줄이는 방식을 제안한다. NFS를 사용함으로 인해 발생하는 오버헤드를 최소화하기 위해 직접 소켓 작업을 할 수 있도록, 정보 기반의 저장 공간 구조 및 메일 저장을 위한 프로토콜을 정의하였다. 제안된 방식은 Netscape에서 만든 Mailstone을 이용하여 실험을 수행하였고, 실험을 통하여 제안된 방식이 기존 방식에 비해 성능이 향상되었음을 확인하였다.

Windows 파일시스템의 디렉토리에 대한 디지털 포렌식 분석 (A Digital Forensic Analysis for Directory in Windows File System)

  • 조규상
    • 디지털산업정보학회논문지
    • /
    • 제11권2호
    • /
    • pp.73-90
    • /
    • 2015
  • When we apply file commands on files in a directory, the directory as well as the file suffer changes in timestamps of MFT entry. Based on understanding of these changes, this work provides a digital forensic analysis on the timestamp changes of the directory influenced by execution of file commands. NTFS utilizes B-tree indexing structure for managing efficient storage of a huge number of files and fast lookups, which changes an index tree of the directory index when files are operated by commands. From a digital forensic point of view, we try to understand behaviors of the B-tree indexes and are looking for traces of files to collect information. But it is not easy to analyze the directory index entry when the file commands are executed. And researches on a digital forensic about NTFS directory and B-tree indexing are comparatively rare. Focusing on the fact, we present, in this paper, directory timestamp changes after executing file commands including a creation, a copy, a deletion etc are analyzed and a method for finding forensic evidences of a deletion of directory containing files. With some cases, i.e. examples of file copy and file deletion command, analyses on the problem of timestamp changes of the directory are given and the problem of finding evidences of a deletion of directory containging files are shown.

A Pattern Summary System Using BLAST for Sequence Analysis

  • Choi, Han-Suk;Kim, Dong-Wook;Ryu, Tae-W.
    • Genomics & Informatics
    • /
    • 제4권4호
    • /
    • pp.173-181
    • /
    • 2006
  • Pattern finding is one of the important tasks in a protein or DNA sequence analysis. Alignment is the widely used technique for finding patterns in sequence analysis. BLAST (Basic Local Alignment Search Tool) is one of the most popularly used tools in bio-informatics to explore available DNA or protein sequence databases. BLAST may generate a huge output for a large sequence data that contains various sequence patterns. However, BLAST does not provide a tool to summarize and analyze the patterns or matched alignments in the BLAST output file. BLAST lacks of general and robust parsing tools to extract the essential information out from its output. This paper presents a pattern summary system which is a powerful and comprehensive tool for discovering pattern structures in huge amount of sequence data in the BLAST. The pattern summary system can identify clusters of patterns, extract the cluster pattern sequences from the subject database of BLAST, and display the clusters graphically to show the distribution of clusters in the subject database.

A bio-text mining system using keywords and patterns in a grid environment

  • Kwon, Hyuk-Ryul;Jung, Tae-Sung;Kim, Kyoung-Ran;Jahng, Hye-Kyoung;Cho, Wan-Sup;Yoo, Jae-Soo
    • 한국산업정보학회:학술대회논문집
    • /
    • 한국산업정보학회 2007년도 춘계학술대회
    • /
    • pp.48-52
    • /
    • 2007
  • As huge amount of literature including biological data is being generated after post genome era, it becomes difficult for researcher to find useful knowledge from the biological databases. Bio-text mining and related natural language processing technique are the key issues in the intelligent knowledge retrieval from the biological databases. We propose a bio-text mining technique for the biologists who find Knowledge from the huge literature. At first, web robot is used to extract and transform related literature from remote databases. To improve retrieval speed, we generate an inverted file for keywords in the literature. Then, text mining system is used for extracting given knowledge patterns and keywords. Finally, we construct a grid computing environment to guarantee processing speed in the text mining even for huge literature databases. In the real experiment for 10,000 bio-literatures, the system shows 95% precision and 98% recall.

  • PDF

Access efficiency of small sized files in Big Data using various Techniques on Hadoop Distributed File System platform

  • Alange, Neeta;Mathur, Anjali
    • International Journal of Computer Science & Network Security
    • /
    • 제21권7호
    • /
    • pp.359-364
    • /
    • 2021
  • In recent years Hadoop usage has been increasing day by day. The need of development of the technology and its specified outcomes are eagerly waiting across globe to adopt speedy access of data. Need of computers and its dependency is increasing day by day. Big data is exponentially growing as the entire world is working in online mode. Large amount of data has been produced which is very difficult to handle and process within a short time. In present situation industries are widely using the Hadoop framework to store, process and produce at the specified time with huge amount of data that has been put on the server. Processing of this huge amount of data having small files & its storage optimization is a big problem. HDFS, Sequence files, HAR, NHAR various techniques have been already proposed. In this paper we have discussed about various existing techniques which are developed for accessing and storing small files efficiently. Out of the various techniques we have specifically tried to implement the HDFS- HAR, NHAR techniques.

Improvement of trajectory tracking control performance by using ILC

  • Le, Dang-Khanh;Nam, Taek-Kun
    • Journal of Advanced Marine Engineering and Technology
    • /
    • 제38권10호
    • /
    • pp.1281-1286
    • /
    • 2014
  • This paper presents an iterative learning control (ILC) approach for tracking problems with specified data points that are desired points at certain time instants. To design ILC systems for such problems, unlike traditional ILC approaches, an algorithm which updates not only the control signal but also the reference trajectory at each trial will be developed. The relationship between the reference trajectory and ILC control in tracking problems where there are specified data points through which the system should pass is investigated as the rate of convergence. In traditional ILC, the desired data is stored in a tracking profile file. Due to the huge size of the data file containing the target points, it is important to reduce the computational cost. Finally, simulation results of the presented technique are mentioned and compared to other related works to confirm the effectiveness of proposed scheme.

Secure Data Sharing in The Cloud Through Enhanced RSA

  • Islam abdalla mohamed;Loay F. Hussein;Anis Ben Aissa;Tarak kallel
    • International Journal of Computer Science & Network Security
    • /
    • 제23권2호
    • /
    • pp.89-95
    • /
    • 2023
  • Cloud computing today provides huge computational resources, storage capacity, and many kinds of data services. Data sharing in the cloud is the practice of exchanging files between various users via cloud technology. The main difficulty with file sharing in the public cloud is maintaining privacy and integrity through data encryption. To address this issue, this paper proposes an Enhanced RSA encryption schema (ERSA) for data sharing in the public cloud that protects privacy and strengthens data integrity. The data owners store their files in the cloud after encrypting the data using the ERSA which combines the RSA algorithm, XOR operation, and SHA-512. This approach can preserve the confidentiality and integrity of a file in any cloud system while data owners are authorized with their unique identities for data access. Furthermore, analysis and experimental results are presented to verify the efficiency and security of the proposed schema.

HBase based Business Process Event Log Schema Design of Hadoop Framework

  • Ham, Seonghun;Ahn, Hyun;Kim, Kwanghoon Pio
    • 인터넷정보학회논문지
    • /
    • 제20권5호
    • /
    • pp.49-55
    • /
    • 2019
  • Organizations design and operate business process models to achieve their goals efficiently and systematically. With the advancement of IT technology, the number of items that computer systems can participate in and the process becomes huge and complicated. This phenomenon created a more complex and subdivide flow of business process.The process instances that contain workcase and events are larger and have more data. This is an essential resource for process mining and is used directly in model discovery, analysis, and improvement of processes. This event log is getting bigger and broader, which leads to problems such as capacity management and I / O load in management of existing row level program or management through a relational database. In this paper, as the event log becomes big data, we have found the problem of management limit based on the existing original file or relational database. Design and apply schemes to archive and analyze large event logs through Hadoop, an open source distributed file system, and HBase, a NoSQL database system.