• Title/Summary/Keyword: Data Filtering

Search Result 1,769, Processing Time 0.032 seconds

Electrooculography Filtering Model Based on Machine Learning (머신러닝 기반의 안전도 데이터 필터링 모델)

  • Hong, Ki Hyeon;Lee, Byung Mun
    • Journal of Korea Multimedia Society
    • /
    • v.24 no.2
    • /
    • pp.274-284
    • /
    • 2021
  • Customized services to a sleep induction for better sleepcare are more effective because of different satisfaction levels to users. The EOG data measured at the frontal lobe when a person blinks his eyes can be used as biometric data because it has different values for each person. The accuracy of measurement is degraded by a noise source, such as toss and turn. Therefore, it is necessary to analyze the noisy data and remove them from normal EOG by filtering. There are low-pass filtering and high-pass filtering as filtering using a frequency band. However, since filtering within a frequency band range is also required for more effective performance, we propose a machine learning model for the filtering of EOG data in this paper as the second filtering method. In addition, optimal values of parameters such as the depth of the hidden layer, the number of nodes of the hidden layer, the activation function, and the dropout were found through experiments, to improve the performance of the machine learning filtering model, and the filtering performance of 95.7% was obtained. Eventually, it is expected that it can be used for effective user identification services by using filtering model for EOG data.

Cooperative Data Stream Filtering for Sensor Tag (센서태그 통합 데이터 필터링에 관한 연구)

  • Ryu, Seung-Wan;Oh, Seul-Ki;Park, Sei-Kwon;Oh, Dong-Ok
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.36 no.8A
    • /
    • pp.683-690
    • /
    • 2011
  • The conventional sensor tag data filtering algorithm uses time window based data filtering for each tag data. However, this approach shows many performance problems such as low error and event detection rate and larger storage size requirement. In this paper, we propose a collaborative sensor tag data filtering algorithm to improve sensor data processing performance. simulation study shows that the proposed sensor tag filtering algorithm outperforms the conventional filtering algorithm in terms of the processing time, the size of required data storage memory and accuracy of error and event detection rate.

An Filtering Automatic Technique of LiDAR Data by Multiple Linear Regression Analysis (다중선형 회귀분석에 의한 LiDAR 자료의 필터링 자동화 기법)

  • Choi, Seung-Pil;Cho, Ji-Hyun;Kim, Jun-Seong
    • Journal of Korean Society for Geospatial Information Science
    • /
    • v.19 no.4
    • /
    • pp.109-118
    • /
    • 2011
  • In this research estimated accuracies that were results in all the area of filtering of the plane equation that was used by whole data set, and regional of filtering that was driven by the plane equation for each vertual Grid. All of this estimates were based by all the area of filtering that deduced the plane equation by multiple linear regression analysis that was used by ground data set. Therefore, accuracy of all the area of filtering that used whole data set has been dropped about 2~3% when average of accuracy of all the area of filtering was based on ground data set while accuracy of Regional of filtering dropped 2~4% when based on virtual Grid. Moreover, as virtual Grid which was set 3~4 cm was difference about 2% of accuracy from standard data. Thus, it leads conclusion of set 3~4 times bigger size in virtual Grid filtering over LiDAR scan gap will be more appropriated. Hence, the result of this research allow us to conclude that there was difference in average accuracy has been noticed when we applied each different approaches, I strongly suggest that it need to research more about real topography for further filtering accuracy.

Improvement of Collaborative Filtering Algorithm Using Imputation Methods

  • Jeong, Hyeong-Chul;Kwak, Min-Jung;Noh, Hyun-Ju
    • Journal of the Korean Data and Information Science Society
    • /
    • v.14 no.3
    • /
    • pp.441-450
    • /
    • 2003
  • Collaborative filtering is one of the most widely used methodologies for recommendation system. Collaborative filtering is based on a data matrix of each customer's preferences and frequently, there exits missing data problem. We introduced two imputation approach (multiple imputation via Markov Chain Monte Carlo method and multiple imputation via bootstrap method) to improve the prediction performance of collaborative filtering and evaluated the performance using EachMovie data.

  • PDF

Data Sparsity and Performance in Collaborative Filtering-based Recommendation

  • Kim Jong-Woo;Lee Hong-Joo
    • Management Science and Financial Engineering
    • /
    • v.11 no.3
    • /
    • pp.19-45
    • /
    • 2005
  • Collaborative filtering is one of the most common methods that e-commerce sites and Internet information services use to personalize recommendations. Collaborative filtering has the advantage of being able to use even sparse evaluation data to predict preference scores for new products. To date, however, no in-depth investigation has been conducted on how the data sparsity effect in customers' evaluation data affects collaborative filtering-based recommendation performance. In this study, we analyzed the sparsity effect and used a hybrid method based on customers' evaluations and purchases collected from an online bookstore. Results indicated that recommendation performance decreased monotonically as sparsity increased, and that performance was more sensitive to sparsity in evaluation data rather than in purchase data. Results also indicated that the hybrid use of two different types of data (customers' evaluations and purchases) helped to improve the recommendation performance when evaluation data were highly sparse.

An Energy Saving Method using Hierarchical Filtering in Sensor Networks (센서 네트워크에서 계층적 필터링을 이용한 에너지 절약 방안)

  • Kim, Jin-Su
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.8 no.4
    • /
    • pp.768-774
    • /
    • 2007
  • This paper proposes how to reduce the amount of data transmitted in each sensor and cluster head in order to lengthen the lifetime of sensor network. This study proposes hierarchical filtering for reducing the sensor's energy dissipation. Hierarchical filtering is to divide sensor network by two tiers when filtering it. First tier performs filtering when transmitting the data from cluster member to cluster head, and second tier performs filtering when transmitting the data from cluster head to base station. This should increase the efficiency of filtering and decrease the inaccuracy of the data compared to the methods which enlarge the filter width to do more filtering.

  • PDF

A Study on Filtering Techniques for Dynamic Analysis of Data Races in Multi-threaded Programs

  • Ha, Ok-Kyoon;Yoo, Hongseok
    • Journal of the Korea Society of Computer and Information
    • /
    • v.22 no.11
    • /
    • pp.1-7
    • /
    • 2017
  • In this paper, we introduce three monitoring filtering techniques which reduce the overheads of dynamic data race detection. It is well known that detecting data races dynamically in multi-threaded programs is quite hard and troublesome task, because the dynamic detection techniques need to monitor all execution of a multi-threaded program and to analyse every conflicting memory and thread operations in the program. Thus, the main drawback of the dynamic analysis for detecting data races is the heavy additional time and space overheads for running the program. For the practicality, we also empirically compare the efficiency of three monitoring filtering techniques. The results using OpenMP benchmarks show that the filtering techniques are practical for dynamic data race detection, since they reduce the average runtime overhead to under 10% of that of the pure detection.

A Model-based Collaborative Filtering Through Regularized Discriminant Analysis Using Market Basket Data

  • Lee, Jong-Seok;Jun, Chi-Hyuck;Lee, Jae-Wook;Kim, Soo-Young
    • Management Science and Financial Engineering
    • /
    • v.12 no.2
    • /
    • pp.71-85
    • /
    • 2006
  • Collaborative filtering, among other recommender systems, has been known as the most successful recommendation technique. However, it requires the user-item rating data, which may not be easily available. As an alternative, some collaborative filtering algorithms have been developed recently by utilizing the market basket data in the form of the binary user-item matrix. Viewing the recommendation scheme as a two-class classification problem, we proposed a new collaborative filtering scheme using a regularized discriminant analysis applied to the binary user-item data. The proposed discriminant model was built in terms of the major principal components and was used for predicting the probability of purchasing a particular item by an active user. The proposed scheme was illustrated with two modified real data sets and its performance was compared with the existing user-based approach in terms of the recommendation precision.

Model of dynamic clustering-based energy-efficient data filtering for mobile RFID networks

  • Vo, Viet Minh Nhat;Le, Van Hoa
    • ETRI Journal
    • /
    • v.43 no.3
    • /
    • pp.427-435
    • /
    • 2021
  • Data filtering is an essential task for improving the energy efficiency of radiofrequency identification (RFID) networks. Among various energy-efficient approaches, clustering-based data filtering is considered to be the most effective solution because data from cluster members can be filtered at cluster heads before being sent to base stations. However, this approach quickly depletes the energy of cluster heads. Furthermore, most previous studies have assumed that readers are fixed and interrogate mobile tags in a workspace. However, there are several applications in which readers are mobile and interrogate fixed tags in a specific area. This article proposes a model for dynamic clustering-based data filtering (DCDF) in mobile RFID networks, where mobile readers are re-clustered periodically and the cluster head role is rotated among the members of each cluster. Simulation results show that DCDF is effective in terms of balancing energy consumption among readers and prolonging the lifetime of the mobile RFID networks.

Enhancing Method to make Cluster for Filtering-based Sensor Networks (여과기법 보안효율을 높이기 위한 센서네트워크 클러스터링 방법)

  • Kim, Byung-Hee;Cho, Tae-Ho
    • 한국정보통신설비학회:학술대회논문집
    • /
    • 2008.08a
    • /
    • pp.141-145
    • /
    • 2008
  • Wireless sensor network (WSN) is expected to be used in many applications. However, sensor nodes still have some secure problems to use them in the real applications. They are typically deployed on open, wide, and unattended environments. An adversary using these features can easily compromise the deployed sensor nodes and use compromised sensor nodes to inject fabricated data to the sensor network (false data injection attack). The injected fabricated data drains much energy of them and causes a false alarm. To detect and drop the injected fabricated data, a filtering-based security method and adaptive methods are proposed. The number of different partitions is important to make event report since they can make a correctness event report if the representative node does not receive message authentication codes made by the different partition keys. The proposed methods cannot guarantee the detection power since they do not consider the filtering scheme. We proposed clustering method for filtering-based secure methods. Our proposed method uses fuzzy system to enhance the detection power of a cluster.

  • PDF