• Title/Summary/Keyword: Outlier Data

Search Result 415, Processing Time 0.028 seconds

Improved LTE Fingerprint Positioning Through Clustering-based Repeater Detection and Outlier Removal

  • Kwon, Jae Uk;Chae, Myeong Seok;Cho, Seong Yun
    • Journal of Positioning, Navigation, and Timing
    • /
    • v.11 no.4
    • /
    • pp.369-379
    • /
    • 2022
  • In weighted k-nearest neighbor (WkNN)-based Fingerprinting positioning step, a process of comparing the requested positioning signal with signal information for each reference point stored in the fingerprint DB is performed. At this time, the higher the number of matched base station identifiers, the higher the possibility that the terminal exists in the corresponding location, and in fact, an additional weight is added to the location in proportion to the number of matching base stations. On the other hand, if the matching number of base stations is small, the selected candidate reference point has high dependence on the similarity value of the signal. But one problem arises here. The positioning signal can be compared with the repeater signal in the signal information stored on the DB, and the corresponding reference point can be selected as a candidate location. The selected reference point is likely to be an outlier, and if a certain weight is applied to the corresponding location, the error of the estimated location information increases. In order to solve this problem, this paper proposes a WkNN technique including an outlier removal function. To this end, it is first determined whether the repeater signal is included in the DB information of the matched base station. If the reference point for the repeater signal is selected as the candidate position, the reference position corresponding to the outlier is removed based on the clustering technique. The performance of the proposed technique is verified through data acquired in Seocho 1 and 2 dongs in Seoul.

A Study on Improving the Reliability of DSRC Traffic Information Considering Traffic and Road Characteristics - Focusing on Busan Urban Expressway - (교통 및 도로특성을 고려한 DSRC 교통정보 신뢰성 향상에 관한 연구)

  • Jeong, Yeon Tak;Jung, Hun Young
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.34 no.5
    • /
    • pp.1535-1545
    • /
    • 2014
  • This study aims at improving the Reliability of DSRC Traffic information considering Traffic and Road Characteristics. First of all, this study analyzed the characteristics of DSRC data on urban expressway and problems of outlier data occurrence. After then, this study produced reliable traffic information by using an optimal method of the Outlier-Filtering. After Outlier-Filtering, this study performed accuracy evaluation and appropriateness check for the number of samples per confidence level. As a result, it showed that the MAPE was between 2.2% and 9.7% and RSME was between 2.2 and 7.5 which are very similar figures to the actual average traffic speed. Also, The samples of both Am peak and Pm peak periods were analyzed to be appropriate at the confidence level of 95%, and 90% within the allowable error range of 5kph.

Outlier detection and treatment in industrial sampling survey (경제조사에서의 이상치 탐지와 처리방법)

  • Joo, Young Sun;Cho, Gyo-Young
    • Journal of the Korean Data and Information Science Society
    • /
    • v.27 no.1
    • /
    • pp.131-142
    • /
    • 2016
  • Outliers in surveys can have a large effect on estimates of totals. This is especially true in business surveys where the populations are drawn are typically skewed. In this paper, we discussed the practical development and implementation of methods to identify and deal with outliers. A detection method is based on quartile method and detected outlier is processed in various ways. The study examines two versions of winsorised estimators with three different cut-off thresholds for each one. For the simulation study, four types of weight transformation function have been considered.

Deep Learning-Based Outlier Detection and Correction for 3D Pose Estimation (3차원 자세 추정을 위한 딥러닝 기반 이상치 검출 및 보정 기법)

  • Ju, Chan-Yang;Park, Ji-Sung;Lee, Dong-Ho
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.11 no.10
    • /
    • pp.419-426
    • /
    • 2022
  • In this paper, we propose a method to improve the accuracy of 3D human pose estimation model in various move motions. Existing human pose estimation models have some problems of jitter, inversion, swap, miss that cause miss coordinates when estimating human poses. These problems cause low accuracy of pose estimation models to detect exact coordinates of human poses. We propose a method that consists of detection and correction methods to handle with these problems. Deep learning-based outlier detection method detects outlier of human pose coordinates in move motion effectively and rule-based correction method corrects the outlier according to a simple rule. We have shown that the proposed method is effective in various motions with the experiments using 2D golf swing motion data and have shown the possibility of expansion from 2D to 3D coordinates.

Robust Location Estimation based on TDOA and FDOA using Outlier Detection Algorithm (이상치 검출 알고리즘을 이용한 TDOA와 FDOA 기반 이동 신호원 위치 추정 기법)

  • Yoo, Hogeun;Lee, Jaehoon
    • Journal of Convergence for Information Technology
    • /
    • v.10 no.9
    • /
    • pp.15-21
    • /
    • 2020
  • This paper presents the outlier detection algorithm in the estimation method of a source location and velocity based on two-step weighted least-squares method using time difference of arrival(TDOA) and frequency difference of arrival(FDOA) data. Since the accuracy of the estimated location and velocity of a moving source can be reduced by the outliers of TDOA and FDOA data, it is important to detect and remove the outliers. In this paper, the method to find the minimum inlier data and the method to determine whether TDOA and FDOA data are included in inliers or outliers are presented. The results of numerical simulations show that the accuracy of the estimated location and velocity is improved by removing the outliers of TDOA and FDOA data.

Correction of Erroneous Individual Vehicle Speed Data Using Locally Weighted Regression (LWR) (국소가중다항회귀분석을 이용한 이상치제거 및 자료보정기법 개발 (GPS를 이용한 개별차량 주행속도를 중심으로))

  • Im, Hui-Seop;O, Cheol;Park, Jun-Hyeong;Lee, Geon-U
    • Journal of Korean Society of Transportation
    • /
    • v.27 no.2
    • /
    • pp.47-56
    • /
    • 2009
  • Effective detection and correction of outliers of raw traffic data collected from the field is of keen interest because reliable traffic information is highly dependent on the quality of raw data. Global positioning system (GPS) based traffic surveillance systems are capable of producing individual vehicle speeds that are invaluable for various traffic management and information strategies. This study proposed a locally weighted regression (LWR) based filtering method for individual vehicle speed data. An important feature of this study was to propose a technique to generate synthetic outliers for more systematic evaluation of the proposed method. It was identified by performance evaluations that the proposed LWR-based method outperformed an exponential smoothing. The proposed method is expected to be effectively utilized for filtering out raw individual vehicle speed data.

The Quartile Deviation and the Control Chart Model of Improvement Confidence for Link Travel Speed from GPS Probe Data (사분위편차 및 관리도 모형에 의한 GPS 수집기반 구간통행속도 데이터 이상치 제거방안 연구)

  • Han, Won-Sub;Kim, Dong-Hyo;Hyun, Cheol-Seung;Lee, Ho-Won;Oh, Yong-Tae;Lee, Choul-Ki
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.7 no.6
    • /
    • pp.21-30
    • /
    • 2008
  • The travel speed collected by the prove-car equipped with the GPS has the problems, which are the data's stability and finding out the representative travel speed, by the influence of the traffic signal and etc. at the interrupted traffic. This study was conducted to develop the method of filtering the outlier data from the data collected by the prove-car. The method to remove the outlier data from the serial data which were collected by the prove-car was adapted to each of the quartile deviation statistics model and the management graphic statistics model. The rate of removing the outlier data by the quartile deviation method was $0{\sim}3.7%$ while the rate by the management graphic statistic methods was $0.3{\sim}7.2%$. Both methods show the low removal rate at the dawn time when the traffic is inactivity, on the other hand the remove rate is high during the daytime. However, both methods have the problem such that the threshold level for removing the outlier data was established at the low bound in the case as good as the statistics model. Therefore, it is required for the experience calibration.

  • PDF

Fault Detection Method for Multivariate Process using Mahalanobis Distance and ICA (마할라노비스 거리와 독립성분분석을 이용한 다변량 공정 고장탐지 방법에 관한 연구)

  • Jung, Seunghwan;Kim, Sungshin
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.14 no.1
    • /
    • pp.22-28
    • /
    • 2021
  • Multivariate processes, such as chemical and mechanical process, power plants are operated in a state where several facilities are complexly connected, the fault of a particular system can also have fatal consequences for the entire process. In addition, since process data is measured in an unstable environment, outlier is likely to be include in the data. Therefore, monitoring technology is essential, which can remove outlier from measured data and detect failures in advance. In this paper, data obtained from dynamic and multivariate process models was used to detect fault in various type of processes. The dynamic process is a simulation of a process with autoregressive property, and the multivariate process is a model that describes a situation when a specific sensor fault. Mahalanobis distance was used to remove outlier contained in the data generated by dynamic process model and multivariate process model, and fault detection was performed using ICA. For comparison, we compared performance with and a conventional single ICA method. The proposed fault detection method improves performance by 0.84%p for bias data and 6.82%p for drift data in the dynamic process. In the case of the multivariate process, the performance was improves by 3.78%p, therefore, the proposed method showed better fault detection performance.

Outlier Detection Method for Mobile Banking with User Input Pattern and E-finance Transaction Pattern (사용자 입력 패턴 및 전자 금융 거래 패턴을 이용한 모바일 뱅킹 이상치 탐지 방법)

  • Min, Hee Yeon;Park, Jin Hyung;Lee, Dong Hoon;Kim, In Seok
    • Journal of Internet Computing and Services
    • /
    • v.15 no.1
    • /
    • pp.157-170
    • /
    • 2014
  • As the increase of transaction using mobile banking continues, threat to the mobile financial security is also increasing. Mobile banking service performs the financial transaction using the dedicate application which is made by financial corporation. It provides the same services as the internet banking service. Personal information such as credit card number, which is stored in the mobile banking application can be used to the additional attack caused by a malicious attack or the loss of the mobile devices. Therefore, in this paper, to cope with the mobile financial accident caused by personal information exposure, we suggest outlier detection method which can judge whether the transaction is conducted by the appropriate user or not. This detection method utilizes the user's input patterns and transaction patterns when a user uses the banking service on the mobile devices. User's input and transaction pattern data involves the information which can be used to discern a certain user. Thus, if these data are utilized appropriately, they can be the information to distinguish abnormal transaction from the transaction done by the appropriate user. In this paper, we collect the data of user's input patterns on a smart phone for the experiment. And we use the experiment data which domestic financial corporation uses to detect outlier as the data of transaction pattern. We verify that our proposal can detect the abnormal transaction efficiently, as a result of detection experiment based on the collected input and transaction pattern data.

Combined Filtering Model Using Voting Rule and Median Absolute Deviation for Travel Time Estimation (통행시간 추정을 위한 Voting Rule과 중위절대편차법 기반의 복합 필터링 모형)

  • Jeong, Youngje;Park, Hyun Suk;Kim, Byung Hwa;Kim, Youngchan
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.12 no.6
    • /
    • pp.10-21
    • /
    • 2013
  • This study suggested combined filtering model to eliminate outlier travel time data in transportation information system, and it was based on Median Absolute Deviation and Voting Rule. This model applied Median Absolute Deviation (MAD) method to follow normal distribution as first filtering process. After that, Voting rule is applied to eliminate remaining outlier travel time data after Median Absolute Deviation. In Voting Rule, travel time samples are judged as outliers according to travel-time difference between sample data and mean data. Elimination or not of outliers are determined using a majority rule. In case study of national highway No. 3, combined filtering model selectively eliminated outliers only and could improve accuracy of estimated travel time.