• Title/Summary/Keyword: 데이터 중복 관리

Search Result 281, Processing Time 0.021 seconds

Storage System Performance Enhancement Using Duplicated Data Management Scheme (중복 데이터 관리 기법을 통한 저장 시스템 성능 개선)

  • Jung, Ho-Min;Ko, Young-Woong
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.37 no.1
    • /
    • pp.8-18
    • /
    • 2010
  • Traditional storage server suffers from duplicated data blocks which cause an waste of storage space and network bandwidth. To address this problem, various de-duplication mechanisms are proposed. Especially, lots of works are limited to backup server that exploits Contents-Defined Chunking (CDC). In backup server, duplicated blocks can be easily traced by using Anchor, therefore CDC scheme is widely used for backup server. In this paper, we propose a new de-duplication mechanism for improving a storage system. We focus on efficient algorithm for supporting general purpose de-duplication server including backup server, P2P server, and FTP server. The key idea is to adapt stride scheme on traditional fixed block duplication checking mechanism. Experimental result shows that the proposed mechanism can minimize computation time for detecting duplicated region of blocks and efficiently manage storage systems.

Data management Scheme modeling for Heterogeneous system integration (이종 시스템 통합을 위한 데이터 관리 기법 모델링)

  • Kang, In-Seong;Lee, Hong-Chul
    • Proceedings of the KAIS Fall Conference
    • /
    • 2010.05a
    • /
    • pp.436-439
    • /
    • 2010
  • 본 논문에서는 Ubiquitous Computing 환경 하에서 이종 시스템 간의 통합을 위한 데이터 관리 기법 모델을 제안하였다. 이종 시스템 간의 통합이 이루어지면 방대한 양의 데이터를 모든 시스템이 공유해야 하기 때문에 무분별한 데이터의 중복과 저장으로 인해 프로세스의 데이터 처리 성능 및 데이터 무결성을 보장받지 못 하는 등의 문제점이 발생한다. 이를 보완하기 위해 Minimal cost Spanning tree의 원리를 적용하여 시스템 통합에 따른 데이터 처리 및 무결성 문제 해결을 위한 메커니즘을 제시하고자 한다.

  • PDF

Data Aggregation for Query Optimization Based on Ocean Sensor Network Architecture (해양 센서 네트워크 아키텍쳐 중심의 질의 최적화를 위한 데이터 병합 기법)

  • Kim, Hae-Jung;Ji, Kyoung-Bok;Kim, Chang-Hwa;Kim, Sang-Kyung;Park, Chan-Jung
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2007.10d
    • /
    • pp.215-220
    • /
    • 2007
  • 최근 센서 네트워크에서 에너지 효율성을 위한 다양한 연구가 진행 중이다. 특히 센서 노드의 저전력을 위해서는 센서 네트워크에서 전송되는 데이터의 횟수나 전송량을 최소한으로 줄이면서 효율적이면서 신뢰성을 가지는 질의에 대한 결과를 얻을 수 있어야 한다. 본 연구에서는 해양 센서 네트워크 상에서 데이터의 전송량을 줄일 수 있는 SDMTree(Sensing Data Management Tree)를 제안한다. 제안된 SDMTree는 질의 최적화를 위해 질의 처리기 구성 요소로 도입 가능하다. 해양 센서 네트워크에서 in-network 각 4레벨에서 하위 노드로부터 받은 데이터를 병합, 관리하기 위한 방법으로 데이터를 속성별로 구분하여 중복된 데이터를 제거하여 트리형태로 구성되기 때문에 질의에 대한 응답에 해당하는 데이터 검색시 정확하고 신속하게 처리할 수 있으며, 트리 구성 또한 중복 데이터 및 중복 영역을 배제하여 구성되므로, 상위노드가 하위 노드로부터 센싱 데이터를 수집하여 저장하기 위한 에너지와 상위 노드에서 하위 노드로 질의를 전송시 질의에 해당하는 특정 영역에만 질의를 전송할 수 있기 때문에 데이터 저장 및 통신에 소모되는 불필요한 에너지를 최대한 줄일 수 있다.

  • PDF

A Case Study on Enhancing Data Quality Through Improvement of Data Management Process: koid Corp (데이터 관리 프로세스 개선을 통한 데이터 품질 개선 사례 연구: (주) 코이드 사례)

  • Huh, Hee-Joung;Kim, Jong-Woo
    • 한국IT서비스학회:학술대회논문집
    • /
    • 2007.11a
    • /
    • pp.345-350
    • /
    • 2007
  • 최근 기업의 각 업무가 정보화 되면서 부문별, 업무별 정보시스템의 데이터 간에 심각한 중복성과 불일치성의 문제가 대두되면서 데이터 품질관리에 관심이 집중되고 있다. 본 연구는 실제로 데이터 표준 관리와 데이터 요구사항 관리를 통매 데이터 품질 관리 프로세스를 개선한 사례를 제시함으로써 데이터 품질 향상을 위해 노력하는 타 기업들에게 도움을 주고자 하였다. 또한, 개선된 데이터 품질 관리 프로세스에 대한 다차원적인 평가로서 데이터 품질, 생산성, 고객만족도, 조직 및 문화의 측면에서 정성 적이고 정량적인 지표를 통한 개선효과를 살펴보고 평가함으로써 제안된 프로세스에 의해 품질수준이 향상되었음을 검증하였고 평가 분석을 통한 시사점을 도출하였다.

  • PDF

Design of Main Memory Database Replication System (주기억장치 데이터베이스 중복 시스템 설계)

  • Choi, Jung-Hyun;Choi, Woo-Young;Jin, Seong-Il;Yeom, Tai-Jin
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2002.11c
    • /
    • pp.1883-1886
    • /
    • 2002
  • 본 논문은 많은 종류의 인터넷 정보시스템에서 데이터의 고속 검색과 저장 및 처리를 지원하는 주기억장치 데이터베이스 시스템을 중복하여 관리할 수 있는 중복 주기억장치 데이터베이스 시스템의 요구사항에 대해 알아본다. 아울러, 자료의 고속처리라는 측면을 주로 고려하여 중복되어 관리되는 주기억장치 데이터베이스 시스템의 아키텍쳐 및 트랜잭션 수행구조를 설계한다.

  • PDF

A study on the application of blockchain technology to prevent duplicate supply and demand of similar welfare services (복지서비스 유사사업의 중복수급 방지를 위한 블록체인 기술 적용 연구)

  • Min, Youn A
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.20 no.6
    • /
    • pp.151-156
    • /
    • 2020
  • Various institutions provide various welfare-related services, and accordingly, the quantity and quality of welfare-related services are improved. Along with the improvement of welfare services, the rate of fraudulent and redundant supply and demand due to inappropriate applicants is also increasing. In particular, in the case of similarly promoted projects by various institutions, confirmation of duplicate applications may be insufficient due to system inconsistency between institutions and delays in qualification verification, which may result in duplicate supply and demand. In this paper, in order to prevent the redundant supply and demand related to welfare service similar projects in various institutions, the problem of data management and sharing between institutions was reviewed and a method of applying blockchain technology was proposed step by step. Through the proposal of this paper, transparent data management of recipients is possible, and through this, trust-based welfare benefit management will be possible.

Efficient Privacy-Preserving Duplicate Elimination in Edge Computing Environment Based on Trusted Execution Environment (신뢰실행환경기반 엣지컴퓨팅 환경에서의 암호문에 대한 효율적 프라이버시 보존 데이터 중복제거)

  • Koo, Dongyoung
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.11 no.9
    • /
    • pp.305-316
    • /
    • 2022
  • With the flood of digital data owing to the Internet of Things and big data, cloud service providers that process and store vast amount of data from multiple users can apply duplicate data elimination technique for efficient data management. The user experience can be improved as the notion of edge computing paradigm is introduced as an extension of the cloud computing to improve problems such as network congestion to a central cloud server and reduced computational efficiency. However, the addition of a new edge device that is not entirely reliable in the edge computing may cause increase in the computational complexity for additional cryptographic operations to preserve data privacy in duplicate identification and elimination process. In this paper, we propose an efficiency-improved duplicate data elimination protocol while preserving data privacy with an optimized user-edge-cloud communication framework by utilizing a trusted execution environment. Direct sharing of secret information between the user and the central cloud server can minimize the computational complexity in edge devices and enables the use of efficient encryption algorithms at the side of cloud service providers. Users also improve the user experience by offloading data to edge devices, enabling duplicate elimination and independent activity. Through experiments, efficiency of the proposed scheme has been analyzed such as up to 78x improvements in computation during data outsourcing process compared to the previous study which does not exploit trusted execution environment in edge computing architecture.

Data Deduplication Method using PRAM Cache in SSD Storage System (SSD 스토리지 시스템에서 PRAM 캐시를 이용한 데이터 중복제거 기법)

  • Kim, Ju-Kyeong;Lee, Seung-Kyu;Kim, Deok-Hwan
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.50 no.4
    • /
    • pp.117-123
    • /
    • 2013
  • In the recent cloud storage environment, the amount of SSD (Solid-State Drive) replacing with the traditional hard disk drive is increasing. Management of SSD for its space efficiency has become important since SSD provides fast IO performance due to no mechanical movement whereas it has wearable characteristics and does not provide in place update. In order to manage space efficiency of SSD, data de-duplication technique is frequently used. However, this technique occurs much overhead because it consists of data chunking, hasing and hash matching operations. In this paper, we propose new data de-duplication method using PRAM cache. The proposed method uses hierarchical hash tables and LRU(Least Recently Used) for data replacement in PRAM. First hash table in DRAM is used to store hash values of data cached in the PRAM and second hash table in PRAM is used to store hash values of data in SSD storage. The method also enhance data reliability against power failure by maintaining backup of first hash table into PRAM. Experimental results show that average writing frequency and operation time of the proposed method are 44.2% and 38.8% less than those of existing data de-depulication method, respectively, when three workloads are used.

A Study of Method to Restore Deduplicated Files in Windows Server 2012 (윈도우 서버 2012에서 데이터 중복 제거 기능이 적용된 파일의 복원 방법에 관한 연구)

  • Son, Gwancheol;Han, Jaehyeok;Lee, Sangjin
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.27 no.6
    • /
    • pp.1373-1383
    • /
    • 2017
  • Deduplication is a function to effectively manage data and improve the efficiency of storage space. When the deduplication is applied to the system, it makes it possible to efficiently use the storage space by dividing the stored file into chunks and storing only unique chunk. However, the commercial digital forensic tool do not support the file system analysis, and the original file extracted by the tool can not be executed or opened. Therefore, in this paper, we analyze the process of generating chunks of data for a Windows Server 2012 system that can apply deduplication, and the structure of the resulting file(Chunk Storage). We also analyzed the case where chunks that are not covered in the previous study are compressed. Based on these results, we propose the method to collect deduplicated data and reconstruct the original file for digital forensic investigation.

A Study on the Collection Use and Overlap of Regional Collaborative Library System in Public Libraries (공공도서관의 지역통합시스템 운영이 장서의 이용 및 중복에 미치는 영향에 관한 연구)

  • Park, Hyun-Kyung;Noh, Dong-Jo
    • Journal of the Korean BIBLIA Society for library and Information Science
    • /
    • v.29 no.4
    • /
    • pp.251-267
    • /
    • 2018
  • In this research, a sample was selected from a public library in Paju city, which operates a regional collaborative library system. They collected data on materials, usage, and acquisition. An analysis of book collection data and usage data was also conducted in order to evaluate the correlation between the usage and acquisition by comparing overlap of purchased materials. As a result, circulations and the interlibrary loan (ILL) borrowing requests were differentiated by topics according to targeted materials, and that these characteristics affect the overlap of purchased materials. The study suggested that public libraries in the region collaborative library system need to consider the paradigm of collection and access when purchasing books.