• Title/Summary/Keyword: Outlier analysis

Search Result 234, Processing Time 0.03 seconds

APPLICATION OF HISTOGRAM OUTLIER ANALYSIS ON THE IMAGE DEGRADATION MODEL FOR BEST FOCAL POINT SELECTION

  • Shin, Hyun-Kyung
    • Journal of applied mathematics & informatics
    • /
    • v.27 no.1_2
    • /
    • pp.175-182
    • /
    • 2009
  • Microscopic imaging system often requires the algorithm to adjust location of camera lenses automatically in machine level. An effort to detect the best focal point is naturally interpreted as a mathematical inverse problem [1]. Following Wiener's point of view [2], we interpret the focus level of images as the quantified factor appeared in image degradation model: g = $f{\ast}H+{\eta}$, a standard mathematical model for understanding signal or image degradation process [3]. In this paper we propose a simple, very fast and robust method to compare the degradation parameters among the multiple images given by introducing outlier analysis of histogram.

  • PDF

A Study on the Outliers Detection in the Number of Railway Passengers for the Gyeongbu Line From Seoul to Major Cities Using a Time Series Outlier Detection Technique (시계열 이상치 탐지 기법을 활용한 경부선 주요도시 철도 승객수의 이상치 탐색 연구)

  • LEE, Jiseon;YOON, Yoonjin
    • Journal of Korean Society of Transportation
    • /
    • v.35 no.6
    • /
    • pp.469-480
    • /
    • 2017
  • On April 1, 2004, KTX (Korea Train eXpress), the first HSR (High-Speed Rail) in Korea, was introduced to Gyeongbu Line. The introduction of the KTX service led to a change in the number of passengers for Gyeongbu Line. Previous studies have analyzed the pre and post-event changes of the intervening events by either simple statistics or intervention ARIMA analysis. However, the intervention ARIMA model has a limitation that several assumptions such as the occurrence time and the type of intervention events are necessary. To this end, this study analyzed the effects of intervention event on the number of passengers using the Gyeongbu line based on a time series outlier detection technique which can overcome limitations in the previous studies. The time series outlier detection technique can analyze the time, effect type and size of an intervention event without the assumption of the time and effect type of the intervention event. The data were collected from the Korea Transport Database (KTDB) for twelve years from 2003 to 2014 (144 months). The analysis results showed that the size of the influence type in the same intervention events was different across the major city routes, and the intervention event which could not be found by previous study methods was also found.

Outlier Analysis of Learner's Learning Behaviors Data using k-NN Method (k-NN 기법을 이용한 학습자의 학습 행위 데이터의 이상치 분석)

  • Yoon, Tae-Bok;Jung, Young-Mo;Lee, Jee-Hyong;Cha, Hyun-Jin;Park, Seon-Hee;Kim, Yong-Se
    • 한국HCI학회:학술대회논문집
    • /
    • 2007.02a
    • /
    • pp.524-529
    • /
    • 2007
  • 지능형 학습 시스템은 학습자의 학습 과정에서 수집된 데이터를 분석하여 학습자에게 맞는 전략을 세우고 적합한 서비스를 제공하는 시스템이다. 학습자에게 적합한 서비스를 위해서는 학습자 모델링 작업이 우선시 되며, 이 모델 생성을 위해서 학습자의 학습 과정에서 발생한 데이터를 수집하고 분석하게 된다. 하지만, 수집된 데이터가 학습자의 일관되지 못한 행위나 비예측 학습 성향을 포함하고 있다면, 생성된 모델을 신뢰하기 어렵다. 본 논문에서는 학습자에게서 수집된 데이터를 거리기반 이상치 선별 방법인 k-NN을 이용하여 이상치를 선별한다. 실험에서는 홈 인테리어 컨텐츠 기반에 학습자의 학습 행위에 대한 학습 성향을 진단하기 위한 DOLLS-HI를 이용하여, 수집된 학습자의 데이터에서 이상치를 분류하고 학습 성향 진단을 위한 모델을 생성하였다. 생성된 모델은 이상치 분류전과 비교하여 신뢰가 향상된 것을 확인하였다.

  • PDF

An outlier weight adjustment using generalized ratio-cum-product method for two phase sampling (이중추출법에서 일반화 ratio-cum-product 방법을 이용한 이상점 가중치 보정법)

  • Oh, Jung-Taek;Shin, Key-Il
    • The Korean Journal of Applied Statistics
    • /
    • v.29 no.7
    • /
    • pp.1185-1199
    • /
    • 2016
  • Two phase sampling (double sampling) is often used when there is inadequate population information for proper stratification. Many recent papers have been devoted to the estimation method to improve the precision of the estimator using first phase information. In this study we suggested outlier weight adjustment methods to improve estimation precision based on the weight of the generalized ratio-cum-product estimator. Small simulation studies are conducted to compare the suggested methods and the usual method. Real data analysis is also performed.

Comparative Analysis on the Outlier Data of Each Parameter in Automatic Water Quality Monitoring Networks (수질자동측정망 자료의 항목별 이상치 비교 분석)

  • Lim, Byungjin;Hong, Eunyoung;Yeon, Insung
    • Journal of Korean Society on Water Environment
    • /
    • v.26 no.4
    • /
    • pp.700-706
    • /
    • 2010
  • Along the 4 major rivers in korea, there are automatic water quality monitoring (AWQM) stations to immediately respond to any pollution incident. Real-time data (temperature, DO, pH, EC and TOC) collected at each station were statistically treated to exclude outliers and keep valid data using Dixon's test and Discordance test. These applied methods were compared in terms of the number of the outliers sorted out. There was no significant difference between these methods. On the other hand, more outliers were sorted out from EC and TOC data, comparing with other water quality items. EC data did not show partly any variation for a long time at H station. If measured signal does not exceed ${\pm}0.001mS/cm$ from the sectional mean, the signal should be treated as normal data. Therefore, another routine was added to the data screening system, some data which were removed as outlier were restored.

Estimation of irrigation supply from agricultural reservoirs based on reservoir storage data

  • Kang, Hansol;An, Hyunuk;Lee, Kwangya
    • Korean Journal of Agricultural Science
    • /
    • v.46 no.4
    • /
    • pp.999-1006
    • /
    • 2019
  • Recently, the quantitative management of agricultural water supply, which is the main source for water consumption in Korea, has become more important due to the effective water management organization of the Korean government. In this study, the estimation method for irrigation supply based on agricultural reservoir storage data was improved compared to previous research, in which drought year selection was unclear, and the outlier data for the rainfall-irrigation supply were not eliminated in the regression analysis. In this study, the drought year was selected by the ratio of annual precipitation to mean annual precipitation and the storage rate observed before the start of irrigation. The outlier data for the rainfall-irrigation supply were eliminated by the Grubbs & Beck test. The proposed method was applied to nine agricultural reservoirs for validation. As a result, the ratio of annual precipitation to mean annual precipitation is less than 53% and the storage rate observed before the start of irrigation is less than 55% it was judged to be the drought year. In addition, the drought supply factor, K, was found to be 0.70 on average, showing closer results to the observed reservoir rates. This shows that water management at the real is appling drought year practice. It was shown that the performance of the proposed method was satisfactory with NSE (Nash-Sutcliffe model efficiency coefficient) and R2 (coefficient of determiniation) except for a few cases.

A study on the outlier data estimation method for anomaly detection of photovoltaic system (태양광 발전 이상감지를 위한 아웃라이어 추정 방법에 대한 연구)

  • Seo, Jong Kwan;Lee, Tae Il;Lee, Whee Sung;Park, Jeom Bae
    • Journal of IKEEE
    • /
    • v.24 no.2
    • /
    • pp.403-408
    • /
    • 2020
  • Photovoltaic (PV) has both intermittent and uncertainty in nature, so it is difficult to accurately predict. Thus anomaly detection technology is important to diagnose real time PV generation. This paper identifies a correlation between various parameters and classifies the PV data applying k-nearest neighbor and dynamic time warpping. Results for the two classifications showed that an outlier detection by a fault of some facilities, and a temporary power loss by partial shading and overall shading occurring during the short period. Based on 100kW plant data, machine learning analysis and test results verified actual outliers and candidates of outlier.

Implementation of Bayesian Filter Method and Range Measurement Analysis for Underwater Robot Localization (수중로봇 위치추정을 위한 베이시안 필터 방법의 실현과 거리 측정 특성 분석)

  • Noh, Sung Woo;Ko, Nak Yong;Kim, Tae Gyun
    • The Journal of Korea Robotics Society
    • /
    • v.9 no.1
    • /
    • pp.28-38
    • /
    • 2014
  • This paper verifies the performance of Extended Kalman Filter(EKF) and MCL(Monte Carlo Localization) approach to localization of an underwater vehicle through experiments. Especially, the experiments use acoustic range sensor whose measurement accuracy and uncertainty is not yet proved. Along with localization, the experiment also discloses the uncertainty features of the range measurement such as bias and variance. The proposed localization method rejects outlier range data and the experiment shows that outlier rejection improves localization performance. It is as expected that the proposed method doesn't yield as precise location as those methods which use high priced DVL(Doppler Velocity Log), IMU(Inertial Measurement Unit), and high accuracy range sensors. However, it is noticeable that the proposed method can achieve the accuracy which is affordable for correction of accumulated dead reckoning error, even though it uses only range data of low reliability and accuracy.

Outlier detection and time series modelling in the stationary time series (정상 시계열에서의 이상치 발견과 시계열 모형구축)

  • 이종협;최기헌
    • The Korean Journal of Applied Statistics
    • /
    • v.5 no.2
    • /
    • pp.139-156
    • /
    • 1992
  • Recently several authors have introduced iterative methods for detecting time series outliers. Most of these methods are developed under the assumption that an underlying outlier-free model is known or can be identified. Since outliers can distort model identification or even make it impossible, we propose procedure begins with a descriptive data analysis of a time series using distance measures between two observations. Properties of the proposed test statistic are presented. To distinguish the type of an outlier are used transfer function models. An empirical example is given to illustrate the time series modeling procedure.

  • PDF

A RSS-Based Localization Method Utilizing Robust Statistics for Wireless Sensor Networks under Non-Gaussian Noise (비 가우시안 잡음이 존재하는 무선 센서 네트워크에서 Robust Statistics를 활용하는 수신신호세기기반의 위치 추정 기법)

  • Ahn, Tae-Joon;Koo, In-Soo
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.11 no.3
    • /
    • pp.23-30
    • /
    • 2011
  • In the wireless sensor network(WSN), the detection of precise location of sensor nodes is essential for efficiently utilizing the sensing data acquired from sensor nodes. Among various location methods, the received signal strength (RSS) based localization scheme is mostly preferable in many applications since it can be easily implemented without any additional hardware cost. Since the RSS localization method is mainly effected by radio channel between two nodes, outlier data can be included in the received signal strength measurement specially when some obstacles move around the link between nodes. The outlier data can have bad effect on estimating the distance between two nodes such that it can cause location errors. In this paper, we propose a RSS-based localization method using Robust Statistic and Gaussian filter algorithm for enhancing the accuracy of RSS-based localization. In the proposed algorithm, the outlier data can be eliminated from samples by using the Robust Statistics as well as the Gaussian filter such that the accuracy of localization can be achieved. Through simulation, it is shown that the proposed algorithm can increase the accuracy of localization and is more robust to non gaussian noise channels.