• Title/Summary/Keyword: 데이터 필터링

Search Result 985, Processing Time 0.026 seconds

Comparative Analysis and Accuracy Improvement on Ground Point Filtering of Airborne LIDAR Data for Forest Terrain Modeling (산림지형 모델링을 위한 항공 라이다 데이터의 지면점 필터링 비교분석과 정확도 개선)

  • Hwang, Se-Ran;Lee, Im-Pyeong
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.29 no.6
    • /
    • pp.641-650
    • /
    • 2011
  • Airborne LIDAR system, utilized in various forest studies, provides efficiently spatial information about vertical structures of forest areas. The tree height is one of the most essential measurements to derive forest information such as biomass, which can be estimated from the forest terrain model. As the terrain model is generated by the interpolation of ground points extracted from LIDAR data, filtering methods with high reliability to classify reliably the ground points are required. In this paper, we applied three representative filtering methods to forest LIDAR data with diverse characteristics, measured the errors and performance of these methods, and analyzed the causes of the errors. Based on their complementary characteristics derived from the analysis results, we have attempted to combine the results and checked the performance improvement. In most test areas, the convergence method showed the satisfactory results, where the filtering performance were improved more than 10% in maximum. Also, we have generated DTM using the classified ground points and compared with the verification data. The DTM retains about 17cm RMSE, which can be sufficiently utilized for the derivation of forest information.

A Study on the Performance Improvement of Software Digital Filter using GPU (GPU를 이용한 소프트웨어 디지털 필터의 성능개선에 관한 연구)

  • Yeom, Jae-Hwan;Oh, Se-Jin;Roh, Duk-Gyoo;Jung, Dong-Kyu;Hwang, Ju-Yeon;Oh, Chungsik;Kim, Hyo-Ryoung
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.19 no.4
    • /
    • pp.153-161
    • /
    • 2018
  • This paper describes the performance improvement of Software (SW) digital filter using GPU (Graphical Processing Unit). The previous developed SW digital filter has a problem that it operates on a CPU (Central Processing Unit) basis and has a slow speed. The GPU was introduced to filter the data of the EAVN (East Asian VLBI Network) observation to improve the operation speed and to process data with other stations through filtering, respectively. In order to enhance the computational speed of the SW digital filter, NVIDIA Titan V GPU board with built-in Tensor Core is used. The processing speed of about 0.78 (1Gbps, 16MHz BW, 16-IF) and 1.1 (2Gbps, 32MHz BW, 16-IF) times for the observing time was achieved by filtering the 95 second observation data of 2 Gbps (512 MHz BW, 1-IF), respectively. In addition, 2Gbps data is digitally filtered for the 1 and 2Gbps simultaneously observed with KVN (Korean VLBI Network), and compared with the 1Gbps, we obtained similar values such as cross power spectrum, phase, and SNR (Signal to Noise Ratio). As a result, the effectiveness of developed SW digital filter using GPU in this research was confirmed for utilizing the data processing and analysis. In the future, it is expected that the observation data will be able to be filtered in real time when the distributed processing optimization of source code for using multiple GPU boards.

Efficient Similarity Joins by Adaptive Prefix Filtering (맞춤 접두 필터링을 이용한 효율적인 유사도 조인)

  • Park, Jong Soo
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.2 no.4
    • /
    • pp.267-272
    • /
    • 2013
  • As an important operation with many applications such as data cleaning and duplicate detection, the similarity join is a challenging issue, which finds all pairs of records whose similarities are above a given threshold in a dataset. We propose a new algorithm that uses the prefix filtering principle as strong constraints on generation of candidate pairs for fast similarity joins. The candidate pair is generated only when the current prefix token of a probing record shares one prefix token of an indexing record within the constrained prefix tokens by the principle. This generation method needs not to compute an upper bound of the overlap between two records, which results in reduction of execution time. Experimental results show that our algorithm significantly outperforms the previous prefix filtering-based algorithms on real datasets.

Talent Recommendation System based-on Personal Propensity and Collaborative Filtering (개인화 요인과 협업적 필터링을 이용한 개인화 재능추천 시스템)

  • Lee, Tae-Su;Seo, Jung-Yeon;Jeon, Eun-Kwang;Lee, Hwa-Min
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2016.10a
    • /
    • pp.481-482
    • /
    • 2016
  • 스마트 디바이스의 발전과 IoT 시대에 들어서면서 다양하고 유용한 데이터가 끊임없이 쏟아져 나오고 있다. 이러한 데이터들을 기반으로 개인화된 추천 시스템의 중요성은 높아지고 있다. 추천 시스템에서 가장 성공적인 협업적 필터링 기법은 고객에 대한 일정 수준 이상의 데이터가 존재해야 한다. 즉, 충분한 데이터가 존재하지 않는다면 정확하지 않은 추천 결과를 출력하는 희박성의 문제가 생긴다. 본 연구에서는 개개인을 구분 지을 수 있는 개인화 요인에 가중치를 두어 기존의 협업적 필터링 기법이 가지는 희박성 문제를 해결하고자 한다.

A Comparative Study on Collaborative Filtering Algorithm (협업 필터링 알고리즘에 관한 비교연구)

  • Li, Jiapei;Li, Xiaomeng;Lee, HyunChang;Shin, SeongYoon
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2017.10a
    • /
    • pp.151-153
    • /
    • 2017
  • In recommendation system, collaborative filtering is the most important algorithm. Collaborative filtering is a method of making automatic predictions about the interests of a user by collecting preferences or taste information from many users. In this paper five algorithms were used. Metrics such as Recall-Precision, FPR-TPR,RMSE, MSE, MAE were calculated. From the result of the experiment, the user-based collaborative filtering was the best approach to recommend movies to users.

  • PDF

Adult Contents Filtering using Speech Information (음성 정보를 이용한 성인 컨텐츠 필터링)

  • Cho, Jung-Ik;Jo, Jin-Su;Lee, Yill-Byung
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2008.05a
    • /
    • pp.145-147
    • /
    • 2008
  • 현재까지 유해한 컨텐츠(Contents)를 차단하기 위한 활발한 연구가 있었으나, 사람의 음성(speech)정보를 이용한 필터링(filtering) 기법에 대한 연구는 활발히 이루어지지 않은 측면이 있다. 본 논문은 동영상 데이터를 가지고 있는 여러 데이터 중에서 음성 정보의 분석을 통하여 일반 컨텐츠와 성인 컨텐츠를 분류하기 위함이다. 본 논문은 음성 정보 중에서 음성 정보의 특징을 가장 잘 다루는 피치 검출을 통한 정보의 분석을 통한 성인 컨텐츠의 필터링에 그 목적이 있다. 현재까지 진행되고 있는 필터링(filtering)방법에 대한 수행 결과보다 개선된 성능을 보이고자 한다. 즉, 음성 정보의 특징 정보를 이용한 성인 컨텐츠(Adult Contents)분류 기법을 활용하는 것으로 성인 컨텐츠(Adult Contents)에서 두드러지는 특징을 보이는 사운드 패턴을 분석하는 것이다.

Implementation and Evaluation of the Sensor Registry System based on Mobile App Profiles (모바일 앱 프로파일 기반 센서 레지스트리 시스템의 구현 및 평가)

  • Choi, Hojin;Yoo, Hyunseok;Jeong, Dongwon;Jeon, Keunhwan
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2016.04a
    • /
    • pp.945-948
    • /
    • 2016
  • 이 논문에서는 기존 센서 레지스트리 시스템의 처리 성능을 개선하기 위해 제안된 모바일 앱 프로파일 기반 센서 필터링 기법을 실험하고 평가한다. 이기종 센서 네트워크 환경에서 센서 레지스트리 시스템은 센서 메타데이터를 제공함으로써 센서 정보의 의미 처리를 즉시적으로 가능하게 한다. 하지만 불필요한 센서 메타데이터까지 처리하여 전체적인 처리 속도를 저하시킨다. 이 문제점을 개선하기 위해 모바일 앱 프로파일 기반 센서 필터링 기법을 제안한다. 모바일 앱 프로파일을 활용하여 센서 데이터의 유효성을 식별 후 모바일 기기에 센서 메타데이터를 기록한다. 기록된 정보를 재사용하여 불필요한 센서 데이터를 필터링 한다. 불필요한 센서 데이터를 처리 하지 않고, 센서 메타 데이터의 요청 횟수를 줄여 전체적인 센서 데이터 처리 속도를 향상 시킨다. 기존 방법과 제안 방법을 구현 하고 실험하여 제안 방법의 전체적인 처리 속도가 향상됨을 확인한다.

Improving Classification Performance for Data with Numeric and Categorical Attributes Using Feature Wrapping (특징 래핑을 통한 숫자형 특징과 범주형 특징이 혼합된 데이터의 클래스 분류 성능 향상 기법)

  • Lee, Jae-Sung;Kim, Dae-Won
    • Journal of KIISE:Software and Applications
    • /
    • v.36 no.12
    • /
    • pp.1024-1027
    • /
    • 2009
  • In this letter, we evaluate the classification performance of mixed numeric and categorical data for comparing the efficiency of feature filtering and feature wrapping. Because the mixed data is composed of numeric and categorical features, the feature selection method was applied to data set after discretizing the numeric features in the given data set. In this study, we choose the feature subset for improving the classification performance of the data set after preprocessing. The experimental result of comparing the classification performance show that the feature wrapping method is more reliable than feature filtering method in the aspect of classification accuracy.

Two-Dimensional Grouping Index for Efficient Processing of XML Filtering Queries (XML 필터링 질의의 효율적 처리를 위한 이차원 그룹핑 색인기법)

  • Yeo, Dae-Hwi;Lee, Jong-Hak
    • Journal of Information Technology and Architecture
    • /
    • v.10 no.1
    • /
    • pp.123-135
    • /
    • 2013
  • This paper presents a two-dimensional grouping index(2DG-index) for efficient processing of XML filtering queries. Recently, many index techniques have been suggested for the efficient processing of structural relationships among the elements in the XML database such as an ancestor- descendant and a parent-child relationship. However, these index techniques focus on simple path queries, and don't consider the path queries that include a condition value for filtering. The 2DG-index is an index structure that deals with the problem of clustering index entries in the twodimensional domain space that consists of a XML path identifier domain and a filtering data value domain. For performance evaluation, we have compared our proposed 2DG-index with the conventional one dimensional index structure such as the data grouping index (DG-index) and the path grouping index (PG-index). As the result of the performance evaluations, we have verified that our proposed 2DG-index can efficiently support the query processing in XML databases according to the query types.

An Active Prefetch Filtering Schemes using Exclusive Prefetch Cache (선인출 전용 캐시를 이용한 적극적 선인출 필터링 기법)

  • Chon Young-Suk;Kim Suk-il;Jeon Joong-nam
    • The KIPS Transactions:PartA
    • /
    • v.12A no.1 s.91
    • /
    • pp.41-52
    • /
    • 2005
  • Memory reference instruction caused by cache miss is the critical factor that limits the processing power of processor. Cache prefetching technique is an effective way to reduce the latency due to memory access. However, excessively aggressive prefetch leads to cache pollution and finally to cancel out the advantage of prefetch. In this study, an active prefetch filtering scheme is introduced which dynamically decides whether to commence prefetching after referring a filtering table to reduce the cache pollution due to unnecessary prefetches. For the precision filtering, an evicted address referencing scheme has been proposed where the filter directly compares the current prefetch address with previous unnecessary prefetch addresses stored in filtering table. Moreover, a small sized exclusive prefetch cache has been introduced to increase the amount of eviction of unnecessarily prefetched addresses to enhance the accuracy of dynamic filtering. The exclusive prefetch cache also prevents useful demand data from being pushed out by prefetched data, while the evicted address direct referencing scheme enables the prefetch cache to keep most of useful prefetch data within its small size. Experimental results from commonly used general and multimedia benchmarks show that the average cache miss ratio has been decreased by $13.3{\%}$ by virtue of enhanced filtering accuracy compared with conventional schemes.