• Title/Summary/Keyword: Huge file system

Search Result 36, Processing Time 0.023 seconds

Metadata Structrues of Huge Shared Disk File System for Large Files in GIS (GIS에서 대용량 파일을 위한 대용량 공유 디스크 파일시스템의 메타데이터 구조)

  • 김경배;이용주;박춘서;신범주
    • Spatial Information Research
    • /
    • v.10 no.1
    • /
    • pp.93-106
    • /
    • 2002
  • The traditional file system are designed to store and manage fur small size files. So. we cannot process the huge files related with geographic information data using the traditional file system such as unix file system or linux file system. In this paper, we propose new metadata structures and management mechanisms for the large file system in geographic information system. The proposed mechanisms use dynamic multi-level mode for large files and dynamic bitmap for huge file system. We implement the proposed mechanisms in the metadata structures of SANtopia is shared disk huge file system for storage area networks(SAN).

  • PDF

Dynamic Bitmap for Huge File System (대용량 파일시스템을 위한 동적 비트맵)

  • Kim, Gyeong-Bae;Lee, Yong-Ju;Park, Chun-Seo;Sin, Beom-Ju
    • The KIPS Transactions:PartA
    • /
    • v.9A no.3
    • /
    • pp.287-294
    • /
    • 2002
  • In this paper we propose a new mechanism for file system using a dynamic bitmap assignment. While traditional file systems rely on a fixed bitmap structures for metadata such as super block, inode, and directory entries, the proposed file system allocates bitmap and allocation area depends on file system features. Our approach gives a solution of the problem that the utilization of the file system depends on the file size in the traditional file systems. We show that the proposed mechanism is superior in the efficiency of disk usage compared to the traditional mechanisms.

A Distributed Architecture Based SMTP Server for Large Email Service (대용량 이메일 서비스를 위한 분산 구조 기반의 SMTP 서버)

  • Kim, Young-Jong;Kwak, Hu-Keun;Chung, Kyu-Sik
    • The KIPS Transactions:PartC
    • /
    • v.16C no.5
    • /
    • pp.597-604
    • /
    • 2009
  • An SMTP(Simple Mail Transfer Protocol) server was designed for delivering and storing user's email across a network. An SMTP server can be distributed as multiple servers for large service, huge users or massive emails. An SMTP server can be constructed by multiple servers with separating file system as email storing space, and each SMTP server can usually share each file system by using the NFS(Network File system). However the NFS is originally designed for sharing each file system across a network, and contains all attributes and features of regular file system. Using this NFS for email storing space of SMTP servers, it makes overhead due to unnecessary work of regular file system. In this paper, we propose a method to do directly operation with socket for reducing work overhead caused by the NFS. For doing directly operation with socket, this paper defines information based storing space structure and a protocol for storing emails. We performed experiments using Mailstone made by Netscape. The experimental results show the performance improvement of the proposed method compared to the existing method.

A Digital Forensic Analysis for Directory in Windows File System (Windows 파일시스템의 디렉토리에 대한 디지털 포렌식 분석)

  • Cho, Gyusang
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.11 no.2
    • /
    • pp.73-90
    • /
    • 2015
  • When we apply file commands on files in a directory, the directory as well as the file suffer changes in timestamps of MFT entry. Based on understanding of these changes, this work provides a digital forensic analysis on the timestamp changes of the directory influenced by execution of file commands. NTFS utilizes B-tree indexing structure for managing efficient storage of a huge number of files and fast lookups, which changes an index tree of the directory index when files are operated by commands. From a digital forensic point of view, we try to understand behaviors of the B-tree indexes and are looking for traces of files to collect information. But it is not easy to analyze the directory index entry when the file commands are executed. And researches on a digital forensic about NTFS directory and B-tree indexing are comparatively rare. Focusing on the fact, we present, in this paper, directory timestamp changes after executing file commands including a creation, a copy, a deletion etc are analyzed and a method for finding forensic evidences of a deletion of directory containing files. With some cases, i.e. examples of file copy and file deletion command, analyses on the problem of timestamp changes of the directory are given and the problem of finding evidences of a deletion of directory containging files are shown.

A Pattern Summary System Using BLAST for Sequence Analysis

  • Choi, Han-Suk;Kim, Dong-Wook;Ryu, Tae-W.
    • Genomics & Informatics
    • /
    • v.4 no.4
    • /
    • pp.173-181
    • /
    • 2006
  • Pattern finding is one of the important tasks in a protein or DNA sequence analysis. Alignment is the widely used technique for finding patterns in sequence analysis. BLAST (Basic Local Alignment Search Tool) is one of the most popularly used tools in bio-informatics to explore available DNA or protein sequence databases. BLAST may generate a huge output for a large sequence data that contains various sequence patterns. However, BLAST does not provide a tool to summarize and analyze the patterns or matched alignments in the BLAST output file. BLAST lacks of general and robust parsing tools to extract the essential information out from its output. This paper presents a pattern summary system which is a powerful and comprehensive tool for discovering pattern structures in huge amount of sequence data in the BLAST. The pattern summary system can identify clusters of patterns, extract the cluster pattern sequences from the subject database of BLAST, and display the clusters graphically to show the distribution of clusters in the subject database.

A bio-text mining system using keywords and patterns in a grid environment

  • Kwon, Hyuk-Ryul;Jung, Tae-Sung;Kim, Kyoung-Ran;Jahng, Hye-Kyoung;Cho, Wan-Sup;Yoo, Jae-Soo
    • Proceedings of the Korea Society for Industrial Systems Conference
    • /
    • 2007.02a
    • /
    • pp.48-52
    • /
    • 2007
  • As huge amount of literature including biological data is being generated after post genome era, it becomes difficult for researcher to find useful knowledge from the biological databases. Bio-text mining and related natural language processing technique are the key issues in the intelligent knowledge retrieval from the biological databases. We propose a bio-text mining technique for the biologists who find Knowledge from the huge literature. At first, web robot is used to extract and transform related literature from remote databases. To improve retrieval speed, we generate an inverted file for keywords in the literature. Then, text mining system is used for extracting given knowledge patterns and keywords. Finally, we construct a grid computing environment to guarantee processing speed in the text mining even for huge literature databases. In the real experiment for 10,000 bio-literatures, the system shows 95% precision and 98% recall.

  • PDF

Access efficiency of small sized files in Big Data using various Techniques on Hadoop Distributed File System platform

  • Alange, Neeta;Mathur, Anjali
    • International Journal of Computer Science & Network Security
    • /
    • v.21 no.7
    • /
    • pp.359-364
    • /
    • 2021
  • In recent years Hadoop usage has been increasing day by day. The need of development of the technology and its specified outcomes are eagerly waiting across globe to adopt speedy access of data. Need of computers and its dependency is increasing day by day. Big data is exponentially growing as the entire world is working in online mode. Large amount of data has been produced which is very difficult to handle and process within a short time. In present situation industries are widely using the Hadoop framework to store, process and produce at the specified time with huge amount of data that has been put on the server. Processing of this huge amount of data having small files & its storage optimization is a big problem. HDFS, Sequence files, HAR, NHAR various techniques have been already proposed. In this paper we have discussed about various existing techniques which are developed for accessing and storing small files efficiently. Out of the various techniques we have specifically tried to implement the HDFS- HAR, NHAR techniques.

Improvement of trajectory tracking control performance by using ILC

  • Le, Dang-Khanh;Nam, Taek-Kun
    • Journal of Advanced Marine Engineering and Technology
    • /
    • v.38 no.10
    • /
    • pp.1281-1286
    • /
    • 2014
  • This paper presents an iterative learning control (ILC) approach for tracking problems with specified data points that are desired points at certain time instants. To design ILC systems for such problems, unlike traditional ILC approaches, an algorithm which updates not only the control signal but also the reference trajectory at each trial will be developed. The relationship between the reference trajectory and ILC control in tracking problems where there are specified data points through which the system should pass is investigated as the rate of convergence. In traditional ILC, the desired data is stored in a tracking profile file. Due to the huge size of the data file containing the target points, it is important to reduce the computational cost. Finally, simulation results of the presented technique are mentioned and compared to other related works to confirm the effectiveness of proposed scheme.

Secure Data Sharing in The Cloud Through Enhanced RSA

  • Islam abdalla mohamed;Loay F. Hussein;Anis Ben Aissa;Tarak kallel
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.2
    • /
    • pp.89-95
    • /
    • 2023
  • Cloud computing today provides huge computational resources, storage capacity, and many kinds of data services. Data sharing in the cloud is the practice of exchanging files between various users via cloud technology. The main difficulty with file sharing in the public cloud is maintaining privacy and integrity through data encryption. To address this issue, this paper proposes an Enhanced RSA encryption schema (ERSA) for data sharing in the public cloud that protects privacy and strengthens data integrity. The data owners store their files in the cloud after encrypting the data using the ERSA which combines the RSA algorithm, XOR operation, and SHA-512. This approach can preserve the confidentiality and integrity of a file in any cloud system while data owners are authorized with their unique identities for data access. Furthermore, analysis and experimental results are presented to verify the efficiency and security of the proposed schema.

HBase based Business Process Event Log Schema Design of Hadoop Framework

  • Ham, Seonghun;Ahn, Hyun;Kim, Kwanghoon Pio
    • Journal of Internet Computing and Services
    • /
    • v.20 no.5
    • /
    • pp.49-55
    • /
    • 2019
  • Organizations design and operate business process models to achieve their goals efficiently and systematically. With the advancement of IT technology, the number of items that computer systems can participate in and the process becomes huge and complicated. This phenomenon created a more complex and subdivide flow of business process.The process instances that contain workcase and events are larger and have more data. This is an essential resource for process mining and is used directly in model discovery, analysis, and improvement of processes. This event log is getting bigger and broader, which leads to problems such as capacity management and I / O load in management of existing row level program or management through a relational database. In this paper, as the event log becomes big data, we have found the problem of management limit based on the existing original file or relational database. Design and apply schemes to archive and analyze large event logs through Hadoop, an open source distributed file system, and HBase, a NoSQL database system.