• Title/Summary/Keyword: Hash Data

Search Result 334, Processing Time 0.025 seconds

Analysis and Evaluation of Frequent Pattern Mining Technique based on Landmark Window (랜드마크 윈도우 기반의 빈발 패턴 마이닝 기법의 분석 및 성능평가)

  • Pyun, Gwangbum;Yun, Unil
    • Journal of Internet Computing and Services
    • /
    • v.15 no.3
    • /
    • pp.101-107
    • /
    • 2014
  • With the development of online service, recent forms of databases have been changed from static database structures to dynamic stream database structures. Previous data mining techniques have been used as tools of decision making such as establishment of marketing strategies and DNA analyses. However, the capability to analyze real-time data more quickly is necessary in the recent interesting areas such as sensor network, robotics, and artificial intelligence. Landmark window-based frequent pattern mining, one of the stream mining approaches, performs mining operations with respect to parts of databases or each transaction of them, instead of all the data. In this paper, we analyze and evaluate the techniques of the well-known landmark window-based frequent pattern mining algorithms, called Lossy counting and hMiner. When Lossy counting mines frequent patterns from a set of new transactions, it performs union operations between the previous and current mining results. hMiner, which is a state-of-the-art algorithm based on the landmark window model, conducts mining operations whenever a new transaction occurs. Since hMiner extracts frequent patterns as soon as a new transaction is entered, we can obtain the latest mining results reflecting real-time information. For this reason, such algorithms are also called online mining approaches. We evaluate and compare the performance of the primitive algorithm, Lossy counting and the latest one, hMiner. As the criteria of our performance analysis, we first consider algorithms' total runtime and average processing time per transaction. In addition, to compare the efficiency of storage structures between them, their maximum memory usage is also evaluated. Lastly, we show how stably the two algorithms conduct their mining works with respect to the databases that feature gradually increasing items. With respect to the evaluation results of mining time and transaction processing, hMiner has higher speed than that of Lossy counting. Since hMiner stores candidate frequent patterns in a hash method, it can directly access candidate frequent patterns. Meanwhile, Lossy counting stores them in a lattice manner; thus, it has to search for multiple nodes in order to access the candidate frequent patterns. On the other hand, hMiner shows worse performance than that of Lossy counting in terms of maximum memory usage. hMiner should have all of the information for candidate frequent patterns to store them to hash's buckets, while Lossy counting stores them, reducing their information by using the lattice method. Since the storage of Lossy counting can share items concurrently included in multiple patterns, its memory usage is more efficient than that of hMiner. However, hMiner presents better efficiency than that of Lossy counting with respect to scalability evaluation due to the following reasons. If the number of items is increased, shared items are decreased in contrast; thereby, Lossy counting's memory efficiency is weakened. Furthermore, if the number of transactions becomes higher, its pruning effect becomes worse. From the experimental results, we can determine that the landmark window-based frequent pattern mining algorithms are suitable for real-time systems although they require a significant amount of memory. Hence, we need to improve their data structures more efficiently in order to utilize them additionally in resource-constrained environments such as WSN(Wireless sensor network).

A Name-based Service Discovering Mechanism for Efficient Service Delivery in IoT (IoT에서 효율적인 서비스 제공을 위한 이름 기반 서비스 탐색 메커니즘)

  • Cho, Kuk-Hyun;Kim, Jung-Jae;Ryu, Minwoo;Cha, Si-Ho
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.19 no.6
    • /
    • pp.46-54
    • /
    • 2018
  • The Internet of Things (IoT) is an environment in which various devices provide services to users through communications. Because of the nature of the IoT, data are stored and distributed in heterogeneous information systems. In this situation, IoT end applications should be able to access data without having information on where the data are or what the type of storage is. This mechanism is called Service Discovery (SD). However, some problems arise, since the current SD architectures search for data in physical devices. First, turnaround time increases from searching for services based on physical location. Second, there is a need for a data structure to manage devices and services separately. These increase the administrator's service configuration complexity. As a result, the device-oriented SD structure is not suitable to the IoT. Therefore, we propose an SD structure called Name-based Service-centric Service Discovery (NSSD). NSSD provides name-based centralized SD and uses the IoT edge gateway as a cache server to speed up service discovery. Simulation results show that NSSD provides about twice the improvement in average turnaround time, compared to existing domain name system and distributed hash table SD architectures.

Effective Load Shedding for Multi-Way windowed Joins Based on the Arrival Order of Tuples on Data Streams (다중 윈도우 조인을 위한 튜플의 도착 순서에 기반한 효과적인 부하 감소 기법)

  • Kwon, Tae-Hyung;Lee, Ki-Yong;Son, Jin-Hyun;Kim, Myoung-Ho
    • Journal of KIISE:Databases
    • /
    • v.37 no.1
    • /
    • pp.1-11
    • /
    • 2010
  • Recently, there has been a growing interest in the processing of continuous queries over multiple data streams. When the arrival rates of tuples exceed the memory capacity of the system, a load shedding technique is used to avoid the system becoming overloaded by dropping some subset of input tuples. In this paper, we propose an effective load shedding algorithm for multi-way windowed joins over multiple data streams. Most previous load shedding algorithms estimate the productivity of each tuple, i.e., the number of join output tuples produced by the tuple, based on its "join attribute value" and drop tuples with the lowest productivity. However, the productivity of a tuple cannot be accurately estimated from its join attribute value when the join attribute values are unique and do not repeat, or the distribution of the join attribute values changes over time. For these cases, we estimate the productivity of a tuple based on its "arrival order" on data streams, rather than its join attribute value. The proposed method can effectively estimate the productivity of a tuple even when the productivity of a tuple cannot be accurately estimated from its join attribute value. Through extensive experiments and analysis, we show that our proposed method outperforms the previous methods in terms of effectiveness and efficiency.

An Improvement in K-NN Graph Construction using re-grouping with Locality Sensitive Hashing on MapReduce (MapReduce 환경에서 재그룹핑을 이용한 Locality Sensitive Hashing 기반의 K-Nearest Neighbor 그래프 생성 알고리즘의 개선)

  • Lee, Inhoe;Oh, Hyesung;Kim, Hyoung-Joo
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.11
    • /
    • pp.681-688
    • /
    • 2015
  • The k nearest neighbor (k-NN) graph construction is an important operation with many web-related applications, including collaborative filtering, similarity search, and many others in data mining and machine learning. Despite its many elegant properties, the brute force k-NN graph construction method has a computational complexity of $O(n^2)$, which is prohibitive for large scale data sets. Thus, (Key, Value)-based distributed framework, MapReduce, is gaining increasingly widespread use in Locality Sensitive Hashing which is efficient for high-dimension and sparse data. Based on the two-stage strategy, we engage the locality sensitive hashing technique to divide users into small subsets, and then calculate similarity between pairs in the small subsets using a brute force method on MapReduce. Specifically, generating a candidate group stage is important since brute-force calculation is performed in the following step. However, existing methods do not prevent large candidate groups. In this paper, we proposed an efficient algorithm for approximate k-NN graph construction by regrouping candidate groups. Experimental results show that our approach is more effective than existing methods in terms of graph accuracy and scan rate.

Development of Software-Defined Perimeter-based Access Control System for Security of Cloud and IoT System (Cloud 및 IoT 시스템의 보안을 위한 소프트웨어 정의 경계기반의 접근제어시스템 개발)

  • Park, Seung-Kyu
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.21 no.2
    • /
    • pp.15-26
    • /
    • 2021
  • Recently, as the introduction of cloud, mobile, and IoT has become active, there is a growing need for technology development that can supplement the limitations of traditional security solutions based on fixed perimeters such as firewalls and Network Access Control (NAC). In response to this, SDP (Software Defined Perimeter) has recently emerged as a new base technology. Unlike existing security technologies, SDP can sets security boundaries (install Gateway S/W) regardless of the location of the protected resources (servers, IoT gateways, etc.) and neutralize most of the network-based hacking attacks that are becoming increasingly sofiscated. In particular, SDP is regarded as a security technology suitable for the cloud and IoT fields. In this study, a new access control system was proposed by combining SDP and hash tree-based large-scale data high-speed signature technology. Through the process authentication function using large-scale data high-speed signature technology, it prevents the threat of unknown malware intruding into the endpoint in advance, and implements a kernel-level security technology that makes it impossible for user-level attacks during the backup and recovery of major data. As a result, endpoint security, which is a weak part of SDP, has been strengthened. The proposed system was developed as a prototype, and the performance test was completed through a test of an authorized testing agency (TTA V&V Test). The SDP-based access control solution is a technology with high potential that can be used in smart car security.

Authentication Protocol of Private Code-based for Advanced Security of RFID System (RFID 시스템 보안 강화를 위한 비공개 코드 기반의 인증 프로토콜)

  • Jang, Bong-Im;Kim, Yong-Tae;Jeong, Yoon-Su;Park, Gil-Cheol
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.14 no.3
    • /
    • pp.737-744
    • /
    • 2010
  • The use of RFID recently tends to increase and is expected to expand all over the industry and life. However, RFID is much vulnerable to the malign threats such as eavesdropping, replay attack, spoofing attack, location tracking in the process of authentication. In particular, it is difficult to apply authentication protocol used in the other previous system to low-priced RFID tag. After all, this paper suggests the scheme of efficient authentication protocol for RFID privacy protection. Compared to the previous scheme, suggested scheme reinforces the checking process of transmission data and is secure from eavesdropping and spoofing attack. It minimizes the operation work of the tag and is very useful to apply to the low-priced tag. It also has the merit to confirm the efficiency of communication by reducing the communication rounds.

A High-speed IP Address Lookup Architecture using Adaptive Multiple Hashing and Prefix Grouping (적응적인 복수 해슁과 프리픽스그룹화를 이용한 고속 IP 주소 검색 구조)

  • Park Hyun-Tae;Moon Byung-In;Kang Sung-Ho
    • Journal of the Institute of Electronics Engineers of Korea TC
    • /
    • v.43 no.5 s.347
    • /
    • pp.137-146
    • /
    • 2006
  • IP address lookup has become a major bottleneck of packet forwarding and a critical issue for high-speed networking techniques in routers. In this paper, we propose an efficient high-speed IP address lookup scheme using adaptive multiple hashing and prefix grouping. According to our analysis results based on routing data distributions, we grouped prefix lengths and selected the number of hash functions in each group adaptively. As a result, we can reduce collisions caused by hashing. Accordingly, a forwarding table of our scheme has good memory efficiency, and thus is organized with the proper number of memory modules. Also, the proposed scheme has the fast building and searching mechanisms to develop the forwarding table only during a single memory access.

Hashing Method with Dynamic Server Information for Load Balancing on a Scalable Cluster of Cache Servers (확장성 있는 캐시 서버 클러스터에서의 부하 분산을 위한 동적 서버 정보 기반의 해싱 기법)

  • Hwak, Hu-Keun;Chung, Kyu-Sik
    • The KIPS Transactions:PartA
    • /
    • v.14A no.5
    • /
    • pp.269-278
    • /
    • 2007
  • Caching in a cache sorrel cluster environment has an advantage that minimizes the request and response tine of internet traffic and web user. Then, one of the methods that increases the hit ratio of cache is using the hash function with cooperative caching. It is keeping a fixed size of the total cache memory regardless of the number of cache servers. On the contrary, if there is no cooperative caching, the total size of cache memory increases proportional to the number of cache sowers since each cache server should keep all the cache data. The disadvantage of hashing method is that clients' requests stress a few servers in all the cache servers due to the characteristics of hashing md the overall performance of a cache server cluster depends on a few servers. In this paper, we propose the method that distributes uniformly client requests between cache servers using dynamic server information. We performed experiments using 16 PCs. Experimental results show the uniform distribution o

Wired.Wireless Internet Service Enabled Custom Bibimbap(Rice Hash) Contents Design (유.무선 인터넷 서비스가 가능한 맞춤형 비빔밥 콘텐츠 설계)

  • Jang, Dai-Ja;Yang, Hye-Jeong;Kwon, Dae-Young;Sung, Woo-Youl
    • The Journal of the Korea Contents Association
    • /
    • v.10 no.7
    • /
    • pp.98-106
    • /
    • 2010
  • In this research a food information contents available as well as for PC web and for mobile web users was designed. The web site providing service for PC web and mobile web clients focuses on offering data on individually matching food based on the member's information. The information obtained from the contents can be practically applied by mobile users helping them to take care of their health. This system focuses on wireless internet mobile users and offers various food menus with cooking videos and also information regarding food supplies for individually matching bibimpab recipes. It gives a good information about one of the most traditional Korean food bibimpab which is made of various kinds of vegetables and greens. It also contains nutrition balance information. Designing the web site as a web site available for wired wireless internet accesses makes it available at any place for mobile wireless internet users so that they can obtain all information regarding products and menu which are good for their health.

AKA-PLA: Enhanced AKA Based on Physical Layer Authentication

  • Yang, Jing;Ji, Xinsheng;Huang, Kaizhi;Yi, Ming;Chen, Yajun
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.7
    • /
    • pp.3747-3765
    • /
    • 2017
  • Existing authentication mechanisms in cellular mobile communication networks are realized in the upper layer by employing cryptographic techniques. Authentication data are broadcasted over the air in plaintext, enabling attackers to completely eavesdrop on the authentication and get some information about the shared secret key between legitimate nodes. Therefore, reusing the same secret key to authenticate several times results in the secret key's information leakage and high attacking rate. In this paper, we consider the most representative authentication mechanism, Authentication and Key Agreement (AKA), in cellular communication networks and propose an enhanced AKA scheme based on Physical Layer Authentication (AKA-PLA). Authentication responses generated by AKA are no longer transmitted in plaintext but masked by wireless channel characteristics, which are not available to adversaries, to generate physical layer authentication responses by a fault-tolerant hash method. The authenticator sets the threshold according to the authentication requirement and channel condition, further verifies the identity of the requester based on the matching result of the physical layer authentication responses. The performance analyses show that the proposed scheme can achieve lower false alarm rate and missing rate, which are a pair of contradictions, than traditional AKA. Besides, it is well compatible with AKA.