• Title/Summary/Keyword: Data Filtering method

Search Result 806, Processing Time 0.033 seconds

Improvement of Collaborative Filtering Algorithm Using Imputation Methods

  • Jeong, Hyeong-Chul;Kwak, Min-Jung;Noh, Hyun-Ju
    • Journal of the Korean Data and Information Science Society
    • /
    • v.14 no.3
    • /
    • pp.441-450
    • /
    • 2003
  • Collaborative filtering is one of the most widely used methodologies for recommendation system. Collaborative filtering is based on a data matrix of each customer's preferences and frequently, there exits missing data problem. We introduced two imputation approach (multiple imputation via Markov Chain Monte Carlo method and multiple imputation via bootstrap method) to improve the prediction performance of collaborative filtering and evaluated the performance using EachMovie data.

  • PDF

Enhancing Method to make Cluster for Filtering-based Sensor Networks (여과기법 보안효율을 높이기 위한 센서네트워크 클러스터링 방법)

  • Kim, Byung-Hee;Cho, Tae-Ho
    • 한국정보통신설비학회:학술대회논문집
    • /
    • 2008.08a
    • /
    • pp.141-145
    • /
    • 2008
  • Wireless sensor network (WSN) is expected to be used in many applications. However, sensor nodes still have some secure problems to use them in the real applications. They are typically deployed on open, wide, and unattended environments. An adversary using these features can easily compromise the deployed sensor nodes and use compromised sensor nodes to inject fabricated data to the sensor network (false data injection attack). The injected fabricated data drains much energy of them and causes a false alarm. To detect and drop the injected fabricated data, a filtering-based security method and adaptive methods are proposed. The number of different partitions is important to make event report since they can make a correctness event report if the representative node does not receive message authentication codes made by the different partition keys. The proposed methods cannot guarantee the detection power since they do not consider the filtering scheme. We proposed clustering method for filtering-based secure methods. Our proposed method uses fuzzy system to enhance the detection power of a cluster.

  • PDF

Optimal Fingerprint Data Filtering Model for Location Based Services (위치기반 서비스 강화를 위한 최적 데이터 필터링 기법 및 측위 시스템 적용 모델)

  • Jung, Jun;Kim, Jae-Hoon
    • Korean Management Science Review
    • /
    • v.29 no.2
    • /
    • pp.79-90
    • /
    • 2012
  • Focusing on the rapid market penetration of smart phones, the importance of LBS (Location Based Service) is drastically increased. However, traditional GPS method has critical weakness caused by limited availability, such as indoor environment. WPS is newly attractive method as a widely applicable positioning method. In WPS, RSSI (Received Signal Strength Indication) data of all Wi-Fi APs (Access Point) are measured and stored into a huge database. The stored RSSI data in database make single radio fingerprint map. By the radio fingerprint map, we can estimate the actual position of target point. The essential factor of radio fingerprint database is data integrity of RSSI. Because of millions of APs in urban area, RSSI measurement data are seriously contaminated. Therefore, we present the unified filtering method for RSSI measurement data. As the results of filtering, we can show the effectiveness of suggested method in practical positioning system of mobile operator.

A Study on Improving Efficiency of Recommendation System Using RFM (RFM을 활용한 추천시스템 효율화 연구)

  • Jeong, Sora;Jin, Seohoon
    • Journal of the Korean Institute of Plant Engineering
    • /
    • v.23 no.4
    • /
    • pp.57-64
    • /
    • 2018
  • User-based collaborative filtering is a method of recommending an item to a user based on the preference of the neighbor users who have similar purchasing history to the target user. User-based collaborative filtering is based on the fact that users are strongly influenced by the opinions of other users with similar interests. Item-based collaborative filtering is a method of recommending an item by comparing the similarity of the user's previously preferred items. In this study, we create a recommendation model using user-based collaborative filtering and item-based collaborative filtering with consumer's consumption data. Collaborative filtering is performed by using RFM (recency, frequency, and monetary) technique with purchasing data to recommend items with high purchase potential. We compared the performance of the recommendation system with the purchase amount and the performance when applying the RFM method. The performance of recommendation system using RFM technique is better.

Monitoring of Chemical Processes Using Modified Scale Space Filtering and Functional-Link-Associative Neural Network (개선된 스케일 스페이스 필터링과 함수연결연상 신경망을 이용한 화학공정 감시)

  • Park, Jung-Hwan;Kim, Yoon-Sik;Chang, Tae-Suk;Yoon, En-Sup
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.6 no.12
    • /
    • pp.1113-1119
    • /
    • 2000
  • To operate a process plant safely and economically, process monitoring is very important. Process monitoring is the task to identify the state of the system from sensor data. Process monitoring includes data acquisition, regulatory control, data reconciliation, fault detection, etc. This research focuses on the data recon-ciliation using scale-space filtering and fault detection using functional-link associative neural networks. Scale-space filtering is a multi-resolution signal analysis method. Scale-space filtering can extract highest frequency factors(noise) effectively. But scale-space filtering has too large calculation costs and end effect problems. This research reduces the calculation cost of scale-space filtering by applying the minimum limit to the gaussian kernel. And the end-effect that occurs at the end of the signal of the scale-space filtering is overcome by using extrapolation related with the clustering change detection method. Nonlinear principal component analysis methods using neural network have been reviewed and the separately expanded functional-link associative neural network is proposed for chemical process monitoring. The separately expanded functional-link associative neural network has better learning capabilities, generalization abilities and short learning time than the exiting-neural networks. Separately expanded functional-link associative neural network can express a statistical model similar to real process by expanding the input data separately. Combining the proposed methods-modified scale-space filtering and fault detection method using the separately expanded functional-link associative neural network-a process monitoring system is proposed in this research. the usefulness of the proposed method is proven by its application a boiler water supply unit.

  • PDF

Electrooculography Filtering Model Based on Machine Learning (머신러닝 기반의 안전도 데이터 필터링 모델)

  • Hong, Ki Hyeon;Lee, Byung Mun
    • Journal of Korea Multimedia Society
    • /
    • v.24 no.2
    • /
    • pp.274-284
    • /
    • 2021
  • Customized services to a sleep induction for better sleepcare are more effective because of different satisfaction levels to users. The EOG data measured at the frontal lobe when a person blinks his eyes can be used as biometric data because it has different values for each person. The accuracy of measurement is degraded by a noise source, such as toss and turn. Therefore, it is necessary to analyze the noisy data and remove them from normal EOG by filtering. There are low-pass filtering and high-pass filtering as filtering using a frequency band. However, since filtering within a frequency band range is also required for more effective performance, we propose a machine learning model for the filtering of EOG data in this paper as the second filtering method. In addition, optimal values of parameters such as the depth of the hidden layer, the number of nodes of the hidden layer, the activation function, and the dropout were found through experiments, to improve the performance of the machine learning filtering model, and the filtering performance of 95.7% was obtained. Eventually, it is expected that it can be used for effective user identification services by using filtering model for EOG data.

A Study on the Context-Aware Reasoning Filtering Mechanism in USN

  • Sung, Kyung;Kim, Seok-Hun;Hong, Min
    • Journal of information and communication convergence engineering
    • /
    • v.9 no.4
    • /
    • pp.452-456
    • /
    • 2011
  • Context-awareness system can provide an optimized services to users. Analyzing physical and complex circumstance elements which give direct or indirect influence to users can tell what users want. However, there are various situation informations around users and it requires high level technology to extract the service what users really want among those informations. The circumstance of the user can be changed from moment to moment, even the service what users want also can be changed in every minutes. Recently the researches to provide the service which a user demands has been progressed actively. Web based filtering method which reaches commercialization is a one of good examples. This method extracts necessary data according to users' demands from the documents on the Web or multimedia informations. However, there is a limit to use it to provide Context-awareness service because it extracts static data, not dynamic data. There is also other researches with a rule based filtering method in progress to filter situation information but this method doesn't have mechanism for dynamic data as well. We would like to solve these problems by providing a dynamic situation information filtering mechanism applying an weighted value about each property of objects and also applying Web based dynamic categories in this paper when unnecessary data should be filtered.

SemFilter: A Simple and Efficient Semantic XML Message Filtering (SemFilter: 단순하며 효율적인 시맨틱 XML 메시지 필터링)

  • Kim, Jae-Hoon;Park, Seog
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.14 no.7
    • /
    • pp.680-693
    • /
    • 2008
  • Recent studies on XML filtering assume that all data sources follow a single global schema defined in a filtering system. However, beyond this simple assumption, a filtering system can provide a service that allows data publishers to have their own schema; hence, the data sources will become heterogeneous. The number of data sources is expected to be large in a filtering system and the data sources are frequently published, updated, and disappeared, that is, dynamic. In this paper, we introduce implementing a simple and efficient XPath query translation method for such a dynamic environment. The method is especially targeted for a query which is composed based only on users' knowledge and experience without a graphical guidance of the global schema. When a user queries a large number of heterogeneous data, there is a high possibility that the query is not consistent with the same local schema assumed by the user. Our query translation method also supports a function for this problem. Some experimental results for query translation performance have shown that our method has reasonable performance, and is more practical than the existing method.

Data BILuring Method for Solving Sparseness Problem in Collaborative Filtering (협동적 여과에서의 희소성 문제 해결을 위한 데이타 블러링 기법)

  • Kim, Hyung-Il;Kim, Jun-Tae
    • Journal of KIISE:Software and Applications
    • /
    • v.32 no.6
    • /
    • pp.542-553
    • /
    • 2005
  • Recommendation systems analyze user preferences and recommend items to a user by predicting the user's preference for those items. Among various kinds of recommendation methods, collaborative filtering(CF) has been widely used and successfully applied to practical applications. However, collaborative filtering has two inherent problems: data sparseness and the cold-start problems. If there are few known preferences for a user, it is difficult to find many similar users, and therefore the performance of recommendation is degraded. This problem is more serious when a new user is first using the system. In this paper we propose a method of integrating additional feature information of users and items into CF to overcome the difficulties caused by sparseness and improve the accuracy of recommendation. In our method, we first fill in unknown preference values by using the probability distribution of feature values, then generate the top-N recommendations by applying collaborative filtering on the modified data. We call this method of filling unknown preference values as data blurring. Several experimental results that show the effectiveness of the proposed method are also presented.

An Effective Filtering Method for Skyline Queries in MANETs (MANET에서 스카이라인 질의를 위한 효과적인 필터링 방법)

  • Park, Mi-Ra;Kim, Min-Kee;Min, Jun-Ki
    • The KIPS Transactions:PartD
    • /
    • v.17D no.4
    • /
    • pp.245-252
    • /
    • 2010
  • In this paper, we propose an effective filtering method for skyline queries in mobile ad hoc networks (MANETs). Most existing researches assume that data is uniformly distributed. Under these assumptions, the previous works focus on optimizing the energy consumption due to the limited battery power. However, in practice, data distribution is skewed in a specific region. In order to reduce the energy consumption, we propose a new filtering method considering the data distribution. We verify the performance of the proposed method through a comparative experiment with an existing method. The results of the experiment confirm that the proposed method reduces the communication overhead and execution time compared to an existing method.