• Title/Summary/Keyword: hash table

Search Result 137, Processing Time 0.028 seconds

An Efficient M-way Stream Join Algorithm Exploiting a Bit-vector Hash Table (비트-벡터 해시 테이블을 이용한 효율적인 다중 스트림 조인 알고리즘)

  • Kwon, Tae-Hyung;Kim, Hyeon-Gyu;Lee, Yu-Won;Kim, Myoung-Ho
    • Journal of KIISE:Databases
    • /
    • v.35 no.4
    • /
    • pp.297-306
    • /
    • 2008
  • MJoin is proposed as an algorithm to join multiple data streams efficiently, whose characteristics are unpredictably changed. It extends a symmetric hash join to handle multiple data streams. Whenever a tuple arrives from a remote stream source, MJoin checks whether all of hash tables have matching tuples. However, when a join involves many data streams with low join selectivity, the performance of this checking process is significantly influenced by the checking order of hash tables. In this paper, we propose a BiHT-Join algorithm which extends MJoin to conduct this checking in a constant time regardless of a join order. BiHT-Join maintains a bit-vector which represents the existence of tuples in streams and decides a successful/unsuccessful join through comparing a bit-vector. Based on the bit-vector comparison, BiHT-Join can conduct a hash join only for successful joining tuples based on this decision. Our experimental results show that the proposed BiHT-Join provides better performance than MJoin in the processing of multiple streams.

Multiple Hashing Architecture using Bloom Filter for IP Address Lookup (IP 주소 검색에서 블룸 필터를 사용한 다중 해싱 구조)

  • Park, Kyong-Hye;Lim, Hye-Sook
    • Journal of KIISE:Databases
    • /
    • v.36 no.2
    • /
    • pp.84-98
    • /
    • 2009
  • Various algorithms and architectures for IP address lookup have been studied to improve forwarding performance in the Internet routers. Previous IP address lookup architecture using Bloom filter requires a separate Bloom filter as well as a separate hash table in each prefix length, and hence it is not efficient in implementation complexity. To reduce the number of hash tables, it applies controlled prefix expansion, but prefix duplication is inevitable in the controlled prefix expansion. Previous parallel multiple-hashing architecture shows very good search performance since it performs parallel search on tables constructed in each prefix length. However, it also has high implementation complexity because of the parallel search structure. In this paper, we propose a new IP address lookup architecture using all-length Bloom filter and all-length multiple hash table, in which various length prefixes are accomodated in a single Bloom filter and a single multiple hash table. Hence the proposed architecture is very good in terms of implementation complexity as well as search performance. Simulation results using actual backbone routing tables which have $15000{\sim}220000$ prefixes show that the proposed architecture requires 1.04-1.17 memory accesses in average for an IP address lookup.

Data Deduplication Method using PRAM Cache in SSD Storage System (SSD 스토리지 시스템에서 PRAM 캐시를 이용한 데이터 중복제거 기법)

  • Kim, Ju-Kyeong;Lee, Seung-Kyu;Kim, Deok-Hwan
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.50 no.4
    • /
    • pp.117-123
    • /
    • 2013
  • In the recent cloud storage environment, the amount of SSD (Solid-State Drive) replacing with the traditional hard disk drive is increasing. Management of SSD for its space efficiency has become important since SSD provides fast IO performance due to no mechanical movement whereas it has wearable characteristics and does not provide in place update. In order to manage space efficiency of SSD, data de-duplication technique is frequently used. However, this technique occurs much overhead because it consists of data chunking, hasing and hash matching operations. In this paper, we propose new data de-duplication method using PRAM cache. The proposed method uses hierarchical hash tables and LRU(Least Recently Used) for data replacement in PRAM. First hash table in DRAM is used to store hash values of data cached in the PRAM and second hash table in PRAM is used to store hash values of data in SSD storage. The method also enhance data reliability against power failure by maintaining backup of first hash table into PRAM. Experimental results show that average writing frequency and operation time of the proposed method are 44.2% and 38.8% less than those of existing data de-depulication method, respectively, when three workloads are used.

An Adaptive Chord for Minimizing Network Traffic in a Mobile P2P Environment (비정기적 데이터 수집 모드에 기반한 효율적인 홈 네트워크 서비스 제어 시스템의 설계)

  • Woo, Hyun-Je;Lee, Mee-Jeong
    • The KIPS Transactions:PartC
    • /
    • v.16C no.6
    • /
    • pp.773-782
    • /
    • 2009
  • A DHT(Distributed Hash Table) based P2P is a method to overcome disadvantages of the existing unstructured P2P method. If a DHT algorithm is used, it can do a fast data search and maintain search efficiency independent of the number of peer. The peers in the DHT method send messages periodically to keep the routing table updated. In a mobile environment, the peers in the DHT method should send messages more frequently to keep the routing table updated and reduce the failure of a request. Therefore, this results in increase of network traffic. In our previous research, we proposed a method to reduce the update load of the routing table in the existing Chord by updating it in a reactive way, but the reactive method had a disadvantage to generate more traffic than the existing Chord if the number of requests per second becomes large. In this paper, we propose an adaptive method of routing table update to reduce the network traffic. In the proposed method, we apply different routing table update method according to the number of request message per second. If the number of request message per second is smaller than some threshold, we apply the reactive method. Otherwsie, we apply the existing Chord method. We perform experiments using Chord simulator (I3) made by UC Berkeley. The experimental results show the performance improvement of the proposed method compared to the existing methods.

An Adaptive Chord for Minimizing Network Traffic in a Mobile P2P Environment (모바일 P2P 환경에서 네트워크 트래픽을 최소화한 적응적인 Chord)

  • Yoon, Young-Hyo;Kwak, Hu-Keun;Kim, Cheong-Ghil;Chung, Kyu-Sik
    • The KIPS Transactions:PartC
    • /
    • v.16C no.6
    • /
    • pp.761-772
    • /
    • 2009
  • A DHT(Distributed Hash Table) based P2P is a method to overcome disadvantages of the existing unstructured P2P method. If a DHT algorithm is used, it can do a fast data search and maintain search efficiency independent of the number of peer. The peers in the DHT method send messages periodically to keep the routing table updated. In a mobile environment, the peers in the DHT method should send messages more frequently to keep the routing table updated and reduce the failure of a request. Therefore, this results in increase of network traffic. In our previous research, we proposed a method to reduce the update load of the routing table in the existing Chord by updating it in a reactive way, but the reactive method had a disadvantage to generate more traffic than the existing Chord if the number of requests per second becomes large. In this paper, we propose an adaptive method of routing table update to reduce the network traffic. In the proposed method, we apply different routing table update method according to the number of request message per second. If the number of request message per second is smaller than some threshold, we apply the reactive method. Otherwsie, we apply the existing Chord method. We perform experiments using Chord simulator (I3) made by UC Berkeley. The experimental results show the performance improvement of the proposed method compared to the existing methods.

Robust Music Identification Using Long-Term Dynamic Modulation Spectrum

  • Kim, Hyoung-Gook;Eom, Ki-Wan
    • The Journal of the Acoustical Society of Korea
    • /
    • v.25 no.2E
    • /
    • pp.69-73
    • /
    • 2006
  • In this paper, we propose a robust music audio fingerprinting system for automatic music retrieval. The fingerprint feature is extracted from the long-term dynamic modulation spectrum (LDMS) estimation in the perceptual compressed domain. The major advantage of this feature is its significant robustness against severe background noise from the street and cars. Further the fast searching is performed by looking up hash table with 32-bit hash values. The hash value bits are quantized from the logarithmic scale modulation frequency coefficients. Experiments illustrate that the LDMS fingerprint has advantages of high scalability, robustness and small fingerprint size. Moreover, the performance is improved remarkably under the severe recording-noise conditions compared with other power spectrum-based robust fingerprints.

Anticipatory I/O Management for Clustered Flash Translation Layer in NAND Flash Memory

  • Park, Kwang-Hee;Yang, Jun-Sik;Chang, Joon-Hyuk;Kim, Deok-Hwan
    • ETRI Journal
    • /
    • v.30 no.6
    • /
    • pp.790-798
    • /
    • 2008
  • Recently, NAND flash memory has emerged as a next generation storage device because it has several advantages, such as low power consumption, shock resistance, and so on. However, it is necessary to use a flash translation layer (FTL) to intermediate between NAND flash memory and conventional file systems because of the unique hardware characteristics of flash memory. This paper proposes a new clustered FTL (CFTL) that uses clustered hash tables and a two-level software cache technique. The CFTL can anticipate consecutive addresses from the host because the clustered hash table uses the locality of reference in a large address space. It also adaptively switches logical addresses to physical addresses in the flash memory by using block mapping, page mapping, and a two-level software cache technique. Furthermore, anticipatory I/O management using continuity counters and a prefetch scheme enables fast address translation. Experimental results show that the proposed address translation mechanism for CFTL provides better performance in address translation and memory space usage than the well-known NAND FTL (NFTL) and adaptive FTL (AFTL).

  • PDF

Clustered Segment Index Scheme for P2P VOD Service on Virtual Mesh Overlay Network (가상 메시 오버레이 네트워크상에서의 P2P VOD 서비스를 위한 클러스터 세그먼트 인덱스 기법)

  • Lim, Pheng-Un;Choi, Hwang-Kyu
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.65 no.6
    • /
    • pp.1052-1059
    • /
    • 2016
  • Video-on-Demand(VoD) is one of the most popular media streaming which attracted many researchers' attention. VMesh is one of the most cited works in the field of the VoD system. VMesh is proposed to solve the problem of random seeking functionality. However, a large number of the DHT(Distributed Hash Table) searches in VMesh is sill the main problem which needs to be solved. In order to reduce the number of the DHT searches, the clustered segment index(CSI) scheme is proposed. In this scheme, the video segments are divided into clusters. The segment information of the video segments, which are clustered into the same cluster, are stored in the same clustered segment index that can be searched by using the hash key. Each peer also can request the required segments by using this clustered segment index. The experiment results show that the number of the DHT searches in the proposed scheme is less than that of VMesh even in case of peers leave and join the network or peers perform the fast forward/backward operations.

An Efficient Algorithm For Mining Association Rules In Main Memory Systems (대용량 주기억장치 시스템에서 효율적인 연관 규칙 탐사 알고리즘)

  • Lee, Jae-Mun
    • The KIPS Transactions:PartD
    • /
    • v.9D no.4
    • /
    • pp.579-586
    • /
    • 2002
  • This paper propose an efficient algorithm for mining association rules in the large main memory systems. To do this, the paper attempts firstly to extend the conventional algorithms such as DHP and Partition in order to be compatible to the large main memory systems and proposes secondly an algorithm to improve Partition algorithm by applying the techniques of the hash table and the bit map. The proposed algorithm is compared to the extended DHP within the experimental environments and the results show up to 65% performance improvement in comparison to the expanded DHP.

Implementation of a Flexible Peer-to-Peer Internet Telephony Service Using an Underlying DHT (유연성을 갖는 분산 해쉬 테이블 기반의 피어 투 피어 인터넷 텔레포니 서비스의 구현)

  • Lee, Ju-Ho;Kim, Jae-Bong;Jeong, Choong-Kyo
    • Journal of Industrial Technology
    • /
    • v.26 no.B
    • /
    • pp.199-206
    • /
    • 2006
  • Internet telephony provides voice communication services with added flexibility for multimedia extension at a lower cost compared to traditional telephone systems. We implemented an internet telephony system as an overlay network without a centralized server, using a distributed hash table (DHT). Compared to the current server-based internet telephony system, our system is fault-tolerant, scalable, and can be flexible extended to various services and advanced to integrated service. To demonstrate the high flexibility of our DHT-based internet telephony system, we made our system cooperate with web servers. Web users can check up others' online stales and establish voice communication sessions to online users at a mouse click. This technology can be applied to more complex services such as multimedia messaging or video conference service.

  • PDF