• Title/Summary/Keyword: Data Filtering method

Search Result 806, Processing Time 0.032 seconds

Efficient Filtering Method for RFID Data Streams (RFID 데이터 스트림의 효율적인 필터링 기법)

  • Yun, Hong-Won
    • The Journal of the Korea Contents Association
    • /
    • v.7 no.10
    • /
    • pp.27-35
    • /
    • 2007
  • Radio Frequency Identification(RFID) technology is set to play an essential role in object tracking or supply chain management systems. New challenges for RFID data management are needed in the RFID applications. RFID data are generated quickly and automatically, and can be used for object tracking, or for real-time monitoring. These applications are mostly associated with the timestamps when the events happen. In this paper, we propose a temporal RFID data model to maintain the history of events and state changes and to monitor the states of RFID objects. Also we propose data filtering method of non active data based on temporal RFID data model. This data model involves essential basic operations for RFID data. We show increased query performance through the data filtering method of non active data.

Study on Performance Improvement of Video in the H.264 Codec (H.264 코덱에서 동영상 성능개선 연구)

  • Bong, Jeong-Sik;Jeon, Joon-Hyeon
    • Proceedings of the KIEE Conference
    • /
    • 2005.10b
    • /
    • pp.532-535
    • /
    • 2005
  • These days, many image processing techniques have been studied for effective image compression. Among those, 2D image filtering is widely used for 2D image processing. The 2D image filtering can be implemented by performing ID linear filtering separately in the direction of horizontal and vertical. Efficiency of image compression depends on what filtering method is used. Generally, circular convolution is widely used in the 2D image filtering for image processing. However it doesn't consider correlations at the region of image boundary, therefore filtering can not be performed effectively. To solve this problem. I proposed new convolution technique using Symmetric-Mirroring convolution, satisfying the 'alias-free' and 'error-free' requirement in the reconstructed image. This method could provide more effective performance than former compression methods. Because it used very high correlative data when performed at the boundary region. In this paper, pre-processing filtering in H.264 codec was adopted to analyze efficiency of proposed filtering technique, and the simulator developed by Matlab language was used to examine the performance of the proposed method.

  • PDF

Data Sparsity and Performance in Collaborative Filtering-based Recommendation

  • Kim Jong-Woo;Lee Hong-Joo
    • Management Science and Financial Engineering
    • /
    • v.11 no.3
    • /
    • pp.19-45
    • /
    • 2005
  • Collaborative filtering is one of the most common methods that e-commerce sites and Internet information services use to personalize recommendations. Collaborative filtering has the advantage of being able to use even sparse evaluation data to predict preference scores for new products. To date, however, no in-depth investigation has been conducted on how the data sparsity effect in customers' evaluation data affects collaborative filtering-based recommendation performance. In this study, we analyzed the sparsity effect and used a hybrid method based on customers' evaluations and purchases collected from an online bookstore. Results indicated that recommendation performance decreased monotonically as sparsity increased, and that performance was more sensitive to sparsity in evaluation data rather than in purchase data. Results also indicated that the hybrid use of two different types of data (customers' evaluations and purchases) helped to improve the recommendation performance when evaluation data were highly sparse.

Data Statical Analysis based Data Filtering Scheme for Monitoring System on Wireless Sensor Network (무선 센서 네트워크 모니터링 시스템을 위한 데이터 통계 분석 기반 데이터 필터링 기법)

  • Lee, Hyun-Jo;Choi, Young-Ho;Chang, Jae-Woo
    • The Journal of the Korea Contents Association
    • /
    • v.10 no.3
    • /
    • pp.53-63
    • /
    • 2010
  • Recently, various monitoring systems are implemented actively by using wireless sensor networks(WSN). When implementing WSN-based monitoring system, there are three important issues to consider. At First, we need to consider a sensor node failure detection method to support the ongoing monitoring. Secondly, because sensor nodes use limited battery power, we need an efficient data filtering method to reduce energy consumption. At Last, a reducing processing overhead method is necessary. The existing Kalman filtering scheme has good performance on data filtering, but it causes too much processing overhead to estimate sensed data. To solve these problems, we, in this paper, propose a new data filtering scheme based on data statical analysis. First, the proposed scheme periodically aggregates node survival massages to support a node failure detection. Secondly, to reduce energy consumption, it sends the sample data with a node survival massage and do data filtering based on those messages. Finally, it analyzes the sample data to estimate filtering range in a server. As a result, each sensor node can use only simple compare operation for filtering data. In addition, we show from our performance analysis that the proposed scheme outperforms the Kalman filtering scheme in terms of the number of sending messages.

Movie Recommendation Algorithm Using Social Network Analysis to Alleviate Cold-Start Problem

  • Xinchang, Khamphaphone;Vilakone, Phonexay;Park, Doo-Soon
    • Journal of Information Processing Systems
    • /
    • v.15 no.3
    • /
    • pp.616-631
    • /
    • 2019
  • With the rapid increase of information on the World Wide Web, finding useful information on the internet has become a major problem. The recommendation system helps users make decisions in complex data areas where the amount of data available is large. There are many methods that have been proposed in the recommender system. Collaborative filtering is a popular method widely used in the recommendation system. However, collaborative filtering methods still have some problems, namely cold-start problem. In this paper, we propose a movie recommendation system by using social network analysis and collaborative filtering to solve this problem associated with collaborative filtering methods. We applied personal propensity of users such as age, gender, and occupation to make relationship matrix between users, and the relationship matrix is applied to cluster user by using community detection based on edge betweenness centrality. Then the recommended system will suggest movies which were previously interested by users in the group to new users. We show shown that the proposed method is a very efficient method using mean absolute error.

Short-term Water Demand Forecasting Algorithm Based on Kalman Filtering with Data Mining (데이터 마이닝과 칼만필터링에 기반한 단기 물 수요예측 알고리즘)

  • Choi, Gee-Seon;Shin, Gang-Wook;Lim, Sang-Heui;Chun, Myung-Geun
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.15 no.10
    • /
    • pp.1056-1061
    • /
    • 2009
  • This paper proposes a short-term water demand forecasting algorithm based on kalman filtering with data mining for sustainable water supply and effective energy saving. The proposed algorithm utilizes a mining method of water supply data and a decision tree method with special days like Chuseok. And the parameters of MLAR (Multi Linear Auto Regression) model are estimated by Kalman filtering algorithm. Thus, we can achieve the practicality of the proposed forecasting algorithm through the good results applied to actual operation data.

Improving Accuracy of Noise Review Filtering for Places with Insufficient Training Data

  • Hyeon Gyu Kim
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.7
    • /
    • pp.19-27
    • /
    • 2023
  • In the process of collecting social reviews, a number of noise reviews irrelevant to a given search keyword can be included in the search results. To filter out such reviews, machine learning can be used. However, if the number of reviews is insufficient for a target place to be analyzed, filtering accuracy can be degraded due to the lack of training data. To resolve this issue, we propose a supervised learning method to improve accuracy of the noise review filtering for the places with insufficient reviews. In the proposed method, training is not performed by an individual place, but by a group including several places with similar characteristics. The classifier obtained through the training can be used for the noise review filtering of an arbitrary place belonging to the group, so the problem of insufficient training data can be resolved. To verify the proposed method, a noise review filtering model was implemented using LSTM and BERT, and filtering accuracy was checked through experiments using real data collected online. The experimental results show that the accuracy of the proposed method was 92.4% on the average, and it provided 87.5% accuracy when targeting places with less than 100 reviews.

An Energy Saving Method using Hierarchical Filtering in Sensor Networks (센서 네트워크에서 계층적 필터링을 이용한 에너지 절약 방안)

  • Kim, Jin-Su
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.8 no.4
    • /
    • pp.768-774
    • /
    • 2007
  • This paper proposes how to reduce the amount of data transmitted in each sensor and cluster head in order to lengthen the lifetime of sensor network. This study proposes hierarchical filtering for reducing the sensor's energy dissipation. Hierarchical filtering is to divide sensor network by two tiers when filtering it. First tier performs filtering when transmitting the data from cluster member to cluster head, and second tier performs filtering when transmitting the data from cluster head to base station. This should increase the efficiency of filtering and decrease the inaccuracy of the data compared to the methods which enlarge the filter width to do more filtering.

  • PDF

Signal Compensation of LiDAR Sensors and Noise Filtering (LiDAR 센서 신호 보정 및 노이즈 필터링 기술 개발)

  • Park, Hong-Sun;Choi, Joon-Ho
    • Journal of Sensor Science and Technology
    • /
    • v.28 no.5
    • /
    • pp.334-339
    • /
    • 2019
  • In this study, we propose a compensation method of raw LiDAR data with noise and noise filtering for signal processing of LiDAR sensors during the development phase. The raw LiDAR data include constant errors generated by delays in transmitting and receiving signals, which can be resolved by LiDAR signal compensation. The signal compensation consists of two stage. First one is LiDAR sensor calibration for a compensation of geometric distortion. Second is walk error compensation. LiDAR data also include fluctuation and outlier noise, the latter of which is removed by data filtering. In this study, we compensate for the fluctuation by using the Kalman filter method, and we remove the outlier noise by applying a Gaussian weight function.

Handling Incomplete Data Problem in Collaborative Filtering System

  • Noh, Hyun-ju;Kwak, Min-jung;Han, In-goo
    • Proceedings of the KAIS Fall Conference
    • /
    • 2003.11a
    • /
    • pp.105-110
    • /
    • 2003
  • Collaborative filtering is one of the methodologies that are most widely used for recommendation system. It is based on a data matrix of each customer's preferences of products. There could be a lot of missing values in such preference. data matrix. This incomplete data is one of the reasons to deteriorate the accuracy of recommendation system. Multiple imputation method imputes m values for each missing value. It overcomes flaws of single imputation approaches through considering the uncertainty of missing values.. The objective of this paper is to suggest multiple imputation-based collaborative filtering approach for recommendation system to improve the accuracy in prediction performance. The experimental works show that the proposed approach provides better performance than the traditional Collaborative filtering approach, especially in case that there are a lot of missing values in dataset used for recommendation system.

  • PDF