• Title/Summary/Keyword: 이상치 발견

Search Result 256, Processing Time 0.028 seconds

Outlier detection and time series modelling in the stationary time series (정상 시계열에서의 이상치 발견과 시계열 모형구축)

  • 이종협;최기헌
    • The Korean Journal of Applied Statistics
    • /
    • v.5 no.2
    • /
    • pp.139-156
    • /
    • 1992
  • Recently several authors have introduced iterative methods for detecting time series outliers. Most of these methods are developed under the assumption that an underlying outlier-free model is known or can be identified. Since outliers can distort model identification or even make it impossible, we propose procedure begins with a descriptive data analysis of a time series using distance measures between two observations. Properties of the proposed test statistic are presented. To distinguish the type of an outlier are used transfer function models. An empirical example is given to illustrate the time series modeling procedure.

  • PDF

A study of a new statistic for detection of outliers and/or influential observations in regression diagnostics (회귀진단에서 이상치와 영향관측치를 동시에 발견하는 새로운 통계량에 관한 연구)

  • 강은미
    • The Korean Journal of Applied Statistics
    • /
    • v.6 no.1
    • /
    • pp.67-78
    • /
    • 1993
  • A new diagnostic statistic for detecting outliers and influential observations in linear models is suggested and studied in this paper. The proposed statistic is a weighted sum of two measures; one is for detecting outliers and the other is for detecting influential observations. The merit of this statistic is that it is possible to distinguish outliers from influential observations. We have done some Monte-Carlo Simulation to find the probability distribution of this statistic.

  • PDF

A Data Mining Tool for Massive Trajectory Data (대규모 궤적 데이타를 위한 데이타 마이닝 툴)

  • Lee, Jae-Gil
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.15 no.3
    • /
    • pp.145-153
    • /
    • 2009
  • Trajectory data are ubiquitous in the real world. Recent progress on satellite, sensor, RFID, video, and wireless technologies has made it possible to systematically track object movements and collect huge amounts of trajectory data. Accordingly, there is an ever-increasing interest in performing data analysis over trajectory data. In this paper, we develop a data mining tool for massive trajectory data. This mining tool supports three operations, clustering, classification, and outlier detection, which are the most widely used ones. Trajectory clustering discovers common movement patterns, trajectory classification predicts the class labels of moving objects based on their trajectories, and trajectory outlier detection finds trajectories that are grossly different from or inconsistent with the remaining set of trajectories. The primary advantage of the mining tool is to take advantage of the information of partial trajectories in the process of data mining. The effectiveness of the mining tool is shown using various real trajectory data sets. We believe that we have provided practical software for trajectory data mining which can be used in many real applications.

Development of quality control techniques for global climate observations (글로벌 기후 관측자료 품질관리 기법 개발)

  • Lee, Jae-Seung;Kim, Seon-Ho;Bae, Deg-Hyo
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2019.05a
    • /
    • pp.104-104
    • /
    • 2019
  • 기후 관측자료의 경우 관측, 가공, 전송 중에 오류가 발생할 수 있으며, 특히 글로벌 기후자료는 다양한 조건을 가지고 있는 자료를 수집하였기 때문에 일반적으로 해당 국가 관측자료보다 품질이 낮다. 본 연구에서는 글로벌 기후 관측자료의 품질을 개선할 수 있는 품질관리 기법을 개발하고 국내 지역에 적용해보고자 한다. 연구대상지역으로 국내 대표도시 7 곳을 선정하였으며, 글로벌 기후자료는 NCDC (National Climatic Data Center)의 일 단위 GSOD (Global Surface Summary of the Day) 자료를 수집하였다. 품질관리는 강수와 기온에 대해서 실시하였으며 과정은 크게 이상치 검사, 이상치 및 결측치 보정, 연, 월 단위 기후 자료 산정으로 구분된다. 이상치 검사는 중복성 검사, 내적일치성 검사, 기후범위 검사, 공간동질성 검사를 기반으로 구성되어 있다. 이상치 및 결측치 보정은 인접 관측소의 자료를 보간하여 수행하였으며, 보간기법은 4 방향 역거리 가중법을 활용하였다. 연, 월 단위 자료 산정은 자료의 결측률을 고려하여 일 단위 자료를 연, 월 단위 자료로 변환하는 과정이다. 이상치 검사 결과 대부분의 이상치는 기후범위와 공간동질성 검사에서 발견되는 것으로 나타났으며, 중복성 및 내적일치성 검사는 이상치 검출 효과가 적은 것으로 나타났다. 결측치 및 이상치 보간 결과 추정된 자료와 관측값 간의 상관관계가 있는 것으로 나타나 활용성이 있었다. 본 연구는 글로벌 자료의 품질관리 기법을 제시하였다는 점에서 활용성이 있으며, 향후 품질관리 기법의 검증에 관한 연구를 수행할 필요가 있다.

  • PDF

Complement Standard Outlier Detection of National Quality Control System Hydrological Data (국가 수문자료 품질관리시스템 이상치 점검 기법 보완)

  • Cho, Herin;Park, Heeseong;Kim, Hyoung Seop
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2017.05a
    • /
    • pp.470-470
    • /
    • 2017
  • 우리나라는 2007년부터 국토교통부 주도하에 관측소별 수문자료 품질관리시스템을 구축하여 강수량, 수위 수문자료를 축척 및 유관기관에 제공하고 있다. 또한 관측소 장비의 오작동 및 주변환경의 영향으로 수문자료에서 이상치 자료가 발견되는 것에 대해서도 자동 및 수동 품질관리 기법을 활용하여 보정하고 있다. 수문자료에 대한 신뢰도와 일관성을 확보하기 위해 이상치 점검 기법에 대한 지속적인 보완 및 개선이 필요하다. 본 연구에서는 기 구축된 수문자료 품질관리시스템에서 이상치 점검에 대한 추가 기법을 조사 및 현 시스템에 적용하여 품질관리의 신뢰도를 향상시키고자 한다. 이를 통해 각 수문자료 관측소에 맞는 품질관리시스템을 지원함으로써 수문자료의 손실을 최소화하고 신뢰도를 향상시키는데 기여할 것으로 기대된다.

  • PDF

Outlier Detection Using Dynamic Plots (동적 그림을 이용한 이상치 검색)

  • Ahn, Byung-Jin;Seo, Han-Son
    • The Korean Journal of Applied Statistics
    • /
    • v.24 no.5
    • /
    • pp.979-986
    • /
    • 2011
  • A linear regression method is commonly used to analyze data because of its simplicity and applicability; however, it is well known that data may contain some outliers and influential cases that may have a harmful effect on a statistical analysis. Thus detection and examination of outliers or influential cases are important parts of data analysis. In detecting multiple outliers, masking effects usually occur and make it difficult to identify the true outliers. We propose to use dynamic plots as a method resistant to masking effect. The procedure using dynamic plots is useful to find appropriate basic sets with which a dependent outliers detection method start and detect a true outliers set. Examples are given to demonstrate the effectiveness of the suggested idea.

Anomaly Detection in Livestock Environmental Time Series Data Using LSTM Autoencoders: A Comparison of Performance Based on Threshold Settings (LSTM 오토인코더를 활용한 축산 환경 시계열 데이터의 이상치 탐지: 경계값 설정에 따른 성능 비교)

  • Se Yeon Chung;Sang Cheol Kim
    • Smart Media Journal
    • /
    • v.13 no.4
    • /
    • pp.48-56
    • /
    • 2024
  • In the livestock industry, detecting environmental outliers and predicting data are crucial tasks. Outliers in livestock environment data, typically gathered through time-series methods, can signal rapid changes in the environment and potential unexpected epidemics. Prompt detection and response to these outliers are essential to minimize stress in livestock and reduce economic losses for farmers by early detection of epidemic conditions. This study employs two methods to experiment and compare performances in setting thresholds that define outliers in livestock environment data outlier detection. The first method is an outlier detection using Mean Squared Error (MSE), and the second is an outlier detection using a Dynamic Threshold, which analyzes variability against the average value of previous data to identify outliers. The MSE-based method demonstrated a 94.98% accuracy rate, while the Dynamic Threshold method, which uses standard deviation, showed superior performance with 99.66% accuracy.

A Study on Outlier Adjustment for Multibeam Echosounder Data (다중빔 음향측심기 자료의 이상치 보정에 관한 연구)

  • Lee, Jung-Sook;Kim, Soo-Young;Lee, Yong-Kook;Shin, Dong-Wan;Jou, Hyeong-Tae;Kim, Han-Joon
    • The Sea:JOURNAL OF THE KOREAN SOCIETY OF OCEANOGRAPHY
    • /
    • v.6 no.1
    • /
    • pp.35-39
    • /
    • 2001
  • Multibeam echosounder data, collected to investigate seabed features and topography, are usually subject to outliers resulting from the ship's irregular movements and insufficient correction for pressure calibration to the positions of beams. We introduce a statistical method which adjusts the outliers using the ARMA (Autoregressive Moving Average) technique. Our method was applied to a set of real data acquired in the East Sea. In our approach, autocorrelation of the data is modeled by an AR (1) model. If an observation is substantially different from that obtained from the estimated AR (1) model, it is declared as an outlier and adjusted using the estimated AR (1) model. This procedure is repeated until no outlier is found. The result of processing shows that outliers that are far greater than signals in amplitude were successfully removed.

  • PDF

THE PREVALENCE OF DOUBLE TEETH AND CONGENITAL MISSING TEETH IN PRIMARY DENTITION AND THEIR CORRELATION WITH THE PERMANENT DENTITION (유치열의 이중치 및 결손치의 발생빈도와 영구치열과의 상호관계)

  • Yang, Kyu-Ho;Lim, Hye-Jeong;Choi, Nam-Ki;Kim, Seon-Mi
    • Journal of the korean academy of Pediatric Dentistry
    • /
    • v.34 no.3
    • /
    • pp.447-453
    • /
    • 2007
  • The purpose of this study was to investigate the relationship between morphology and number of deciduous teeth and the occurrence of other dental anomalies in their successors, and to evaluate the necessity of early diagnosis of dental anomalies in the primary dentition. Prevalence of double teeth and congenital missing teeth was investigated in 254(134 boys, 120 girls) panoramic radiographic films, taken by 2 to 7-year-old children in Chonnam National University Hospital from 2000 to 2005. And then it was examined that relationship of anomalies of the primary dentition and their successors. Among them 11 children(6 boys, 5 girls) had double teeth or missing teeth. And prevalence of the double teeth was 1.6% and missing teeth was 3.1%. One subject had double teeth in in the mandible and missing teeth in the maxilla. Of the 11 cases of dental anomalies in primary dentition, 7 cases had congenital missing tooth in their successors. This study suggests that the dental anomalies in the primary dentition induced high prevalence of the congenital missing of permanent successors in the permanent dentition.

  • PDF

Clinical Management and Short-term Prognosis of Molar-Incisor Malformation Affected Patients: Case Reports (대구치-절치 형태이상 환자의 임상적 치료 및 단기 예후: 증례 보고)

  • Kim, Hyojin;Lim, Sumin;Kim, JinYoung
    • Journal of the korean academy of Pediatric Dentistry
    • /
    • v.49 no.1
    • /
    • pp.121-130
    • /
    • 2022
  • Molar-incisor malformation (MIM) is a newly reported dental anomaly with molar root deformity and incisor crown defects. MIM-affected teeth may cause severe pain with no apparent tooth caries. Since the affected molars clinically appear normal, radiographs are recommended for accurate diagnosis on the first visit. Since MIM-affected patients are in mixed dentition, timely and appropriate interventions are needed to avoid unnecessary pain and complicated clinical issues. This report was written to describe two patients who had MIM in early mixed dentition and report their 2-year follow-ups.