• Title/Summary/Keyword: Data filtering

Search Result 1,780, Processing Time 0.025 seconds

Short-term Water Demand Forecasting Algorithm Based on Kalman Filtering with Data Mining (데이터 마이닝과 칼만필터링에 기반한 단기 물 수요예측 알고리즘)

  • Choi, Gee-Seon;Shin, Gang-Wook;Lim, Sang-Heui;Chun, Myung-Geun
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.15 no.10
    • /
    • pp.1056-1061
    • /
    • 2009
  • This paper proposes a short-term water demand forecasting algorithm based on kalman filtering with data mining for sustainable water supply and effective energy saving. The proposed algorithm utilizes a mining method of water supply data and a decision tree method with special days like Chuseok. And the parameters of MLAR (Multi Linear Auto Regression) model are estimated by Kalman filtering algorithm. Thus, we can achieve the practicality of the proposed forecasting algorithm through the good results applied to actual operation data.

Development of a Personalized Similarity Measure using Genetic Algorithms for Collaborative Filtering

  • Lee, Soojung
    • Journal of the Korea Society of Computer and Information
    • /
    • v.23 no.12
    • /
    • pp.219-226
    • /
    • 2018
  • Collaborative filtering has been most popular approach to recommend items in online recommender systems. However, collaborative filtering is known to suffer from data sparsity problem. As a simple way to overcome this problem in literature, Jaccard index has been adopted to combine with the existing similarity measures. We analyze performance of such combination in various data environments. We also find optimal weights of factors in the combination using a genetic algorithm to formulate a similarity measure. Furthermore, optimal weights are searched for each user independently, in order to reflect each user's different rating behavior. Performance of the resulting personalized similarity measure is examined using two datasets with different data characteristics. It presents overall superiority to previous measures in terms of recommendation and prediction qualities regardless of the characteristics of the data environment.

Efficient Filtering Method for RFID Data Streams (RFID 데이터 스트림의 효율적인 필터링 기법)

  • Yun, Hong-Won
    • The Journal of the Korea Contents Association
    • /
    • v.7 no.10
    • /
    • pp.27-35
    • /
    • 2007
  • Radio Frequency Identification(RFID) technology is set to play an essential role in object tracking or supply chain management systems. New challenges for RFID data management are needed in the RFID applications. RFID data are generated quickly and automatically, and can be used for object tracking, or for real-time monitoring. These applications are mostly associated with the timestamps when the events happen. In this paper, we propose a temporal RFID data model to maintain the history of events and state changes and to monitor the states of RFID objects. Also we propose data filtering method of non active data based on temporal RFID data model. This data model involves essential basic operations for RFID data. We show increased query performance through the data filtering method of non active data.

Design of RFID Air Protocol Filtering and Probabilistic Simulation of Identification Procedure (RFID 무선 프로토콜 필터링의 설계와 확률적 인식 과정 시뮬레이션)

  • Park, Hyun-Sung;Kim, Jong-Deok
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.34 no.6B
    • /
    • pp.585-594
    • /
    • 2009
  • Efficient filtering is an important factor in RFID system performance. Because of huge volume of tag data in future ubiquitous environment, if RFID readers transmit tag data without filtering to upper-layer applications, which results in a significant system performance degradation. In this paper, we provide an efficient filtering technique which operates on RFID air protocol. RFID air protocol filtering between tags and a reader has some advantages over filtering in readers and middleware, because air protocol filtering reduces the volume of filtering work before readers and middleware start filtering. Exploiting the air protocol filtering advantage, we introduce a geometrical algorithm for generating air protocol filters and verify their performance through simulation with analytical time models. Results of dense RFID reader environment show that air protocol filtering algorithms reduce almost a half of the total filtering time when compared to the results of linear search.

Design of a High-Speed RFID Filtering Engine and Cache Based Improvement (고속 RFID 필터링 엔진의 설계와 캐쉬 기반 성능 향상)

  • Park Hyun-Sung;Kim Jong-Deok
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.31 no.5A
    • /
    • pp.517-525
    • /
    • 2006
  • In this paper, we present a high-speed RFID data filtering engine designed to carry out filtering under the conditions of massive data and massive filters. We discovered that the high-speed RFID data filtering technique is very similar to the high-speed packet classification technique which is used in high-speed routers and firewall systems. Actually, our filtering engine is designed based on existing packet classification algorithms, Bit Parallelism and Aggregated Bit Vector(ABV). In addition, we also discovered that there are strong temporal relations and redundancy in the RFID data filtering operations. We incorporated two kinds of caches, tag and filter caches, to make use of this characteristic to improve the efficiency of the filtering engine. The performance of the proposed engine has been examined by implementing a prototype system and testing it. Compared to the basic sequential filter comparison approach, our engine shows much better performance, and it gets better as the number of filters increases.

Weighted Window Assisted User History Based Recommendation System (가중 윈도우를 통한 사용자 이력 기반 추천 시스템)

  • Hwang, Sungmin;Sokasane, Rajashree;Tri, Hiep Tuan Nguyen;Kim, Kyungbaek
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.4 no.6
    • /
    • pp.253-260
    • /
    • 2015
  • When we buy items in online stores, it is common to face recommended items that meet our interest. These recommendation system help users not only to find out related items, but also find new things that may interest users. Recommendation system has been widely studied and various models has been suggested such as, collaborative filtering and content-based filtering. Though collaborative filtering shows good performance for predicting users preference, there are some conditions where collaborative filtering cannot be applied. Sparsity in user data causes problems in comparing users. Systems which are newly starting or companies having small number of users are also hard to apply collaborative filtering. Content-based filtering should be used to support this conditions, but content-based filtering has some drawbacks and weakness which are tendency of recommending similar items, and keeping history of a user makes recommendation simple and not able to follow up users preference changes. To overcome this drawbacks and limitations, we suggest weighted window assisted user history based recommendation system, which captures user's purchase patterns and applies them to window weight adjustment. The system is capable of following current preference of a user, removing useless recommendation and suggesting items which cannot be simply found by users. To examine the performance under user and data sparsity environment, we applied data from start-up trading company. Through the experiments, we evaluate the operation of the proposed recommendation system.

SemFilter: A Simple and Efficient Semantic XML Message Filtering (SemFilter: 단순하며 효율적인 시맨틱 XML 메시지 필터링)

  • Kim, Jae-Hoon;Park, Seog
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.14 no.7
    • /
    • pp.680-693
    • /
    • 2008
  • Recent studies on XML filtering assume that all data sources follow a single global schema defined in a filtering system. However, beyond this simple assumption, a filtering system can provide a service that allows data publishers to have their own schema; hence, the data sources will become heterogeneous. The number of data sources is expected to be large in a filtering system and the data sources are frequently published, updated, and disappeared, that is, dynamic. In this paper, we introduce implementing a simple and efficient XPath query translation method for such a dynamic environment. The method is especially targeted for a query which is composed based only on users' knowledge and experience without a graphical guidance of the global schema. When a user queries a large number of heterogeneous data, there is a high possibility that the query is not consistent with the same local schema assumed by the user. Our query translation method also supports a function for this problem. Some experimental results for query translation performance have shown that our method has reasonable performance, and is more practical than the existing method.

A Study on the Context-Aware Reasoning Filtering Mechanism in USN

  • Sung, Kyung;Kim, Seok-Hun;Hong, Min
    • Journal of information and communication convergence engineering
    • /
    • v.9 no.4
    • /
    • pp.452-456
    • /
    • 2011
  • Context-awareness system can provide an optimized services to users. Analyzing physical and complex circumstance elements which give direct or indirect influence to users can tell what users want. However, there are various situation informations around users and it requires high level technology to extract the service what users really want among those informations. The circumstance of the user can be changed from moment to moment, even the service what users want also can be changed in every minutes. Recently the researches to provide the service which a user demands has been progressed actively. Web based filtering method which reaches commercialization is a one of good examples. This method extracts necessary data according to users' demands from the documents on the Web or multimedia informations. However, there is a limit to use it to provide Context-awareness service because it extracts static data, not dynamic data. There is also other researches with a rule based filtering method in progress to filter situation information but this method doesn't have mechanism for dynamic data as well. We would like to solve these problems by providing a dynamic situation information filtering mechanism applying an weighted value about each property of objects and also applying Web based dynamic categories in this paper when unnecessary data should be filtered.

Probability Adjustment Scheme for the Dynamic Filtering in Wireless Sensor Networks Using Fuzzy Logic (무선 센서 네트워크에서 동적 여과를 위한 퍼지 기반 확률 조절 기법)

  • Han, Man-Ho;Lee, Hae-Young;Cho, Tae-Ho
    • 한국정보통신설비학회:학술대회논문집
    • /
    • 2008.08a
    • /
    • pp.159-162
    • /
    • 2008
  • Generally, sensor nodes can be easily compromised and seized by an adversary because sensor nodes are hostile environments after dissemination. An adversary may be various security attacks into the networks using compromised node. False data injection attack using compromised node, it may not only cause false alarms, but also the depletion of the severe amount of energy waste. Dynamic en-route scheme for Filtering False Data Injection (DEF) can detect and drop such forged report during the forwarding process. In this scheme, each forwarding nodes verify reports using a regular probability. In this paper, we propose verification probability adjustment scheme of forwarding nodes though a fuzzy rule-base system for the Dynamic en-route filtering scheme for Filtering False Data Injection in sensor networks. Verification probability determination of forwarding nodes use false traffic rate and distance form source to base station.

  • PDF

Filtering Effect in Supervised Classification of Polarimetric Ground Based SAR Images

  • Kang, Moon-Kyung;Kim, Kwang-Eun;Cho, Seong-Jun;Lee, Hoon-Yol;Lee, Jae-Hee
    • Korean Journal of Remote Sensing
    • /
    • v.26 no.6
    • /
    • pp.705-719
    • /
    • 2010
  • We investigated the speckle filtering effect in supervised classification of the C-band polarimetric Ground Based SAR image data. Wishart classification method was used for the supervised classification of the polarimetric GB-SAR image data and total of 6 kinds of speckle filters were applied before supervised classification, which are boxcar, Gaussian, Lopez, IDAN, the refined Lee, and the refined Lee sigma filters. For each filters, we changed the filtering kernel size from $3{\times}3$ to $9{\times}9$ to investigate the filtering size effect also. The refined Lee filter with the kernel size of bigger than $5{\times}5$ showed the best result for the Wishart supervised classification of polarimetric GB-SAR image data. The result also showed that the type of trees could be discriminated by Wishart supervised classification of polarimetric GB-SAR image data.