• Title/Summary/Keyword: Distributed File Systems

Search Result 123, Processing Time 0.126 seconds

Scalable Blockchain Storage Model Based on DHT and IPFS

  • Chen, Lu;Zhang, Xin;Sun, Zhixin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.7
    • /
    • pp.2286-2304
    • /
    • 2022
  • Blockchain is a distributed ledger that combines technologies such as cryptography, consensus mechanism, peer-to-peer transmission, and time stamping. The rapid development of blockchain has attracted attention from all walks of life, but storage scalability issues have hindered the application of blockchain. In this paper, a scalable blockchain storage model based on Distributed Hash Table (DHT) and the InterPlanetary File System (IPFS) was proposed. This paper introduces the current research status of the scalable blockchain storage model, as well as the basic principles of DHT and the InterPlanetary File System. The model construction and workflow are explained in detail. At the same time, the DHT network construction mechanism, block heat identification mechanism, new node initialization mechanism, and block data read and write mechanism in the model are described in detail. Experimental results show that this model can reduce the storage burden of nodes, and at the same time, the blockchain network can accommodate more local blocks under the same block height.

Web-based Draft Verification System for Injection Mold Design (사출금형설계를 위한 웹기반 구배 검증 시스템)

  • Yeon Kwang-Heum;Song In-Ho;Chung Sung-Chong
    • Transactions of the Korean Society of Mechanical Engineers A
    • /
    • v.29 no.10 s.241
    • /
    • pp.1353-1360
    • /
    • 2005
  • Injection-molded products serve a wide range of applications in our modem lives and their significance is ever increasing. However, difficulty of communication among related companies under the present system results in increase of lead time and decrease of production efficiency. The objective of this paper is the development of a web-based draft verification system in mold design processes. Although several commercial CAD systems offer draft verification functions, those systems are very expensive and inadequate to perform collaborative works. For collaborative work under the distributed environment, the proposed system uses native file transforming of CAD data into optimal format by using the ACIS kernel and InterOp. Functions of draft verification modules are constructed over the ActiveX control using the visual C++ and OpenGL. Therefore, collaborators related to the development of a new product are able to verify the draft and undercut over the Internet without commercial CAD systems. The system helps to reduce production cost, errors and lead-time to the market. Performance of the system is confirmed through various case studies.

An XML-based Digital Mock-Up System for Heterogeneous Multi-CAD Assembly (XML을 이용한 이기종 CAD 조립체 DMU시스템의 설계)

  • Song, In-Ho;Chung, Sung-Chong
    • Transactions of the Korean Society of Mechanical Engineers A
    • /
    • v.31 no.6 s.261
    • /
    • pp.635-643
    • /
    • 2007
  • As many engineers and technicians are involved in the design process of large scale and/or complex products, there are a lot of miss matches and interferences due to designers' faults and several kinds of CAD systems. Recently, CAD systems are applied to verify and check the assembly process. Digital Mock-Up(DMU) system, a tool to build a virtual mock-up in the design stage, has been used to prevent the interferences and miss matches during precision design processes. Using the virtual assembly tool, engineers are able to design precision and interference free parts without physical mock-ups. Instead of a single CAD source, several CAD systems are used to design a complex product. Several organizations are involved in the distributed design environment for heterogeneous multi-CAD assembly. XML and the lightweight CAD file are proposed for the multi-CAD assembly. XML data contains hierarchy of the heterogenenous multi-CAD assembly. STEP PDM schema and STEP ISO 10303-28 formations are applied to construct the XML data. The lightweight CAD file produced from various CAD files through ACIS kernel and InterOp not only contains mesn, B-Rep and topological data, but also is used to visualize CAD data and to verify dimensions. Developed system is executed on the desktop computers. It does not require commercial CAD systems to visualize 3D assembly data. Real-time interference and fitness checks, dimensional verification, and design and assembly verification are performed on the developed system. Assembly of heterogeneous models for a car is conducted to verify the effectiveness of the developed DMU system on the Internet.

Performance Optimization in GlusterFS on SSDs (SSD 환경 아래에서 GlusterFS 성능 최적화)

  • Kim, Deoksang;Eom, Hyeonsang;Yeom, Heonyoung
    • KIISE Transactions on Computing Practices
    • /
    • v.22 no.2
    • /
    • pp.95-100
    • /
    • 2016
  • In the current era of big data and cloud computing, the amount of data utilized is increasing, and various systems to process this big data rapidly are being developed. A distributed file system is often used to store the data, and glusterFS is one of popular distributed file systems. As computer technology has advanced, NAND flash SSDs (Solid State Drives), which are high performance storage devices, have become cheaper. For this reason, datacenter operators attempt to use SSDs in their systems. They also try to install glusterFS on SSDs. However, since the glusterFS is designed to use HDDs (Hard Disk Drives), when SSDs are used instead of HDDs, the performance is degraded due to structural problems. The problems include the use of I/O-cache, Read-ahead, and Write-behind Translators. By removing these features that do not fit SSDs which are advantageous for random I/O, we have achieved performance improvements, by up to 255% in the case of 4KB random reads, and by up to 50% in the case of 64KB random reads.

Storage Space Page Grouping in a Distributed Local File System on Desktop Systems to Optimize File Availability (데스크탑 시스템을 이용한 분산 지역 파일 시스템에서의 파일 가용도 최적화를 위한 저장 공간 페이지 배치 기법)

  • 김준우;신현식;장래혁
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2001.10c
    • /
    • pp.856-858
    • /
    • 2001
  • 분산 파일 시스템에서 파일을 복제하는 목적의 하나는 파일의 가용성을 높이는 것이다 서버들로 이루어진 시스템에서 가용도 문제는 주로 오류 상황이 대상이다 반면에 데스크탑으로 구성된 경우 각각이 상대적으로 가용도가 낮고 그 편차도 커서 파일의 가용도를 최적화 하도록 복제본을 저장하는 것이 기본적인 문제가 된다. 파일 단위 복제 기법에서는 파일 수가 참여 시스템 수에 따라 기하급수적으로 증가하고 파일 배치 때마다 가용도 최적화를 수행해야 하는 문제가 있다. 본 논문에서는 저장 공간 단위 복제 기법을 제안한다. 데스크탑 시스템의 저 장롱간을 고정 크기의 페이지들로 나누고 이들을 조합해서 일정한 가용도를 가지는 페이지 그룹을 형성한다. 파일들은 이 페이지 그룹에 저장된다. 페이지의 개수가 파일 수가 아닌 시스템 수에 의존하고, 파일 배치 시점과 무관하게 페이지 그룹을 만들 수 있는 장점이 있다. 최종 가용도를 높이고 연산 시간을 줄이기 위해 ILS(Iterated Local Search )방법을 이용만 탐색을 수행한다.

  • PDF

Design and Implementation of a Metadata Structure for Large-Scale Shared-Disk File System (대용량 공유디스크 파일 시스템에 적합한 메타 데이타 구조의 설계 및 구현)

  • 이용주;김경배;신범주
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.30 no.1
    • /
    • pp.33-49
    • /
    • 2003
  • Recently, there have been large storage demands for manipulating multimedia data. To solve the tremendous storage demands, one of the major researches is the SAN(Storage Area Network) that provides the local file requests directly from shared-disk storage and also eliminates the server bottlenecks to performance and availability. SAN also improve the network latency and bandwidth through new channel interface like FC(Fibre Channel). But to manipulate the efficient storage network like SAN, traditional local file system and distributed file system are not adaptable and also are lack of researches in terms of a metadata structure for large-scale inode object such as file and directory. In this paper, we describe the architecture and design issues of our shared-disk file system and provide the efficient bitmap for providing the well-formed block allocation in each host, extent-based semi flat structure for storing large-scale file data, and two-phase directory structure of using Extendible Hashing. Also we describe a detailed algorithm for implementing the file system's device driver in Linux Kernel and compare our file system with the general file system like EXT2 and shard disk file system like GFS in terms of file creation, directory creation and I/O rate.

Randomized Block Size (RBS) Model for Secure Data Storage in Distributed Server

  • Sinha, Keshav;Paul, Partha;Amritanjali, Amritanjali
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.12
    • /
    • pp.4508-4530
    • /
    • 2021
  • Today distributed data storage service are being widely used. However lack of proper means of security makes the user data vulnerable. In this work, we propose a Randomized Block Size (RBS) model for secure data storage in distributed environments. The model work with multifold block sizes encrypted with the Chinese Remainder Theorem-based RSA (C-RSA) technique for end-to-end security of multimedia data. The proposed RBS model has a key generation phase (KGP) for constructing asymmetric keys, and a rand generation phase (RGP) for applying optimal asymmetric encryption padding (OAEP) to the original message. The experimental results obtained with text and image files show that the post encryption file size is not much affected, and data is efficiently encrypted while storing at the distributed storage server (DSS). The parameters such as ciphertext size, encryption time, and throughput have been considered for performance evaluation, whereas statistical analysis like similarity measurement, correlation coefficient, histogram, and entropy analysis uses to check image pixels deviation. The number of pixels change rate (NPCR) and unified averaged changed intensity (UACI) were used to check the strength of the proposed encryption technique. The proposed model is robust with high resilience against eavesdropping, insider attack, and chosen-plaintext attack.

Cost-Effective Replication Schemes for Query Load Balancing in DHT-Based Peer-to-Peer File Searches

  • Cao, Qi;Fujita, Satoshi
    • Journal of Information Processing Systems
    • /
    • v.10 no.4
    • /
    • pp.628-645
    • /
    • 2014
  • In past few years, distributed hash table (DHT)-based P2P systems have been proven to be a promising way to manage decentralized index information and provide efficient lookup services. However, the skewness of users' preferences regarding keywords contained in a multi-keyword query causes a query load imbalance that combines both routing and response load. This imbalance means long file retrieval latency that negatively influences the overall system performance. Although index replication has a great potential for alleviating this problem, existing schemes did not explicitly address it or incurred high cost. To overcome this issue, we propose, in this paper, an integrated solution that consists of three replication schemes to alleviate query load imbalance while minimizing the cost. The first scheme is an active index replication that is used in order to decrease routing load in the system and to distribute response load of an index among peers that store replicas of the index. The second scheme is a proactive pointer replication that places location information of each index to a predetermined number of peers for reducing maintenance cost between the index and its replicas. The third scheme is a passive index replication that guarantees the maximum query load of peers. The result of simulations indicates that the proposed schemes can help alleviate the query load imbalance of peers. Moreover, it was found by comparison that our schemes are more cost-effective on placing replicas than PCache and EAD.

Design of GlusterFS Based Big Data Distributed Processing System in Smart Factory (스마트 팩토리 환경에서의 GlusterFS 기반 빅데이터 분산 처리 시스템 설계)

  • Lee, Hyeop-Geon;Kim, Young-Woon;Kim, Ki-Young;Choi, Jong-Seok
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.11 no.1
    • /
    • pp.70-75
    • /
    • 2018
  • Smart Factory is an intelligent factory that can enhance productivity, quality, customer satisfaction, etc. by applying information and communications technology to the entire production process including design & development, manufacture, and distribution & logistics. The precise amount of data generated in a smart factory varies depending on the factory's size and state of facilities. Regardless, it would be difficult to apply traditional production management systems to a smart factory environment, as it generates vast amounts of data. For this reason, the need for a distributed big-data processing system has risen, which can process a large amount of data. Therefore, this article has designed a Gluster File System (GlusterFS)-based distributed big-data processing system that can be used in a smart factory environment. Compared to existing distributed processing systems, the proposed distributed big-data processing system reduces the system load and the risk of data loss through the distribution and management of network traffic.

DNS-based Dynamic Load Balancing Method on a Distributed Web-server System (분산 웹 서버 시스템에서의 DNS 기반 동적 부하분산 기법)

  • Moon, Jong-Bae;Kim, Myung-Ho
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.33 no.3
    • /
    • pp.193-204
    • /
    • 2006
  • In most existing distributed Web systems, incoming requests are distributed to servers via Domain Name System (DNS). Although such systems are simple to implement, the address caching mechanism easily results in load unbalancing among servers. Moreover, modification of the DNS is necessary to load considering the server's state. In this paper, we propose a new dynamic load balancing method using dynamic DNS update and round-robin mechanism. The proposed method performs effective load balancing without modification of the DNS. In this method, a server can dynamically be added to or removed from the DNS list according to the server's load. By removing the overloaded server from the DNS list, the response time becomes faster. For dynamic scheduling, we propose a scheduling algorithm that considers the CPU, memory, and network usage. We can select a scheduling policy based on resources usage. The proposed system can easily be managed by a GUI-based management tool. Experiments show that modules implemented in this paper have low impact on the proposed system. Furthermore, experiments show that both the response time and the file transfer rate of the proposed system are faster than those of a pure Round-Robin DNS.