• Title/Summary/Keyword: regression outlier

Search Result 116, Processing Time 0.021 seconds

Development of a Observational Settlement Analysis Method Using Outliers (이상치를 이용한 관측적 침하예측기법의 개발)

  • 우철웅;장병욱
    • Magazine of the Korean Society of Agricultural Engineers
    • /
    • v.45 no.5
    • /
    • pp.140-150
    • /
    • 2003
  • Observational methods such as the Asaoka's method and the hyperbolic method are widely applied on the settlement analysis using observed settlement. The most unreliable aspects in those methods is arose from the subjective discretion of initial non-linearity on linear regression. The initial non-linearity is inevitable due to the settlement behaviour itself. Therefore an objective method is essential to achieve more reliable results on settlement analysis. It was found that the initial non-linear data are statistical outliers. New automation algorithms of the hyperbolic and the Asaoka's method were developed based on outlier detection method. The methods are a successive detection of outliers and a searching method of suitable hyperbolic range for the Asaoka's and the hyperbolic method respectively. Applicability of the algorithms was verified through case studies.

Calibration by Median Regression

  • Jinsan Yang;Lee, Seung-Ho
    • Journal of the Korean Statistical Society
    • /
    • v.28 no.2
    • /
    • pp.265-277
    • /
    • 1999
  • Classical and inverse estimation methods are two well known methods in statistical calibration problems. When there are outliers, both methods have large MSE's and could not estimate the input value correctly. We suggest median calibration estimation based on the LD-statistics. To investigate the robust performances, the influence function of the median calibration estimator is calculated and compared with other methods. When there are outliers in the response variables, the influence function is found to be bounded. In simulation studies, the MSE's for each calibration methods are compared. The estimated inputs as well as the performance of the influence functions are calculated.

  • PDF

Lowess and outlier analysis of biological oxygen demand on Nakdong main stream river (낙동강 본류 측정소들의 생물학적 산소요구량 수치에 대한 비모수적 회귀분석과 특이점분석)

  • Kim, Jong Tae
    • Journal of the Korean Data and Information Science Society
    • /
    • v.25 no.1
    • /
    • pp.119-130
    • /
    • 2014
  • This paper is based on water information system of NIE, National Institute of Environmental Research. We used monthly data of water quality from January, 2013 to August, 2013 starting from measuring point A (nbA) to measuring point N (nbN) located along the Nakdong river main stream. Statistical water quality analysis of BOD (biological oxygen demand) is specified by R programming depending on month, year, and points. Based on BOD measured from Nakdong river's measuring points, we used exploratory data analysis and locally weighted scatter plot smoother (Lowess) trend analysis, which is a method of non-parametic regression analysis, to analyze long-term water tendency and water quality distribution depending on points. Also, we analyzed the period and the measuring point of which the outliers are abundant. As a result, compared to BOD measured in nbM located in Busan along the downstream, BOD measured in nbG located in Daegu and nbI located in Changwon along the midstream showed higher rate of water pollution at a severe level.

Fingerprint Segmentation and Ridge Orientation Estimation with a Mobile Camera for Fingerprint Recognition (모바일 카메라를 이용한 지문인식을 위한 지문영역 추출 및 융선방향 추출 알고리즘)

  • Lee Chulhan;Lee Sanghoon;Kim Jaihie;Kim Sung-Jae
    • Journal of the Institute of Electronics Engineers of Korea SP
    • /
    • v.42 no.6
    • /
    • pp.89-98
    • /
    • 2005
  • Fingerprint segmentation and ridge orientation estimation algorithms with images from a mobile camera are proposed. The fingerprint images from a mobile camera are quite different from those from conventional sensor, called touch based sensor such as optical, capacitive, and thermal. For example, the images from a mobile camera are colored and the backgrounds or non-finger regions are very erratic depending on how the image capture time and place. Also the contrast between ridge and valley of a mobile camera image are lower than that of touch based sensor image. To segment fingerprint region, we first detect the initial region using color information and texture information. The LUT (Look Up Table) is used to model the color distribution of fingerprint images using manually segmented images and frequency information is extracted to discriminate between in focused fingerprint regions and out of focused background regions. With the detected initial region, the region growing algerian is executed to segment final fingerprint region. In fingerprint orientation estimation, the problem of gradient based method is very sensitive to outlier that occurred by scar and camera noise. To solve this problem, we propose a robust regression method that removes the outlier iteratively and effectively. In the experiments, we evaluated the result of the proposed fingerprint segmentation algerian using 600 manually segmented images and compared the orientation algorithms in terms of recognition accuracy.

A Comparison of Calibration Methods for the COCOMO II Post-Architecture Model (COCOMOII의 후구조 모델에 대한 캘리브레이션 방법 비교)

  • Yoon, Myoung-Young
    • Proceedings of the Korea Society for Industrial Systems Conference
    • /
    • 2000.05a
    • /
    • pp.135-143
    • /
    • 2000
  • The COCOMO Ⅱ model is well-suited for the new software development life cycle such as non-sequential and rapid-development processes. The traditional regression approach based on the least square criterion is the most commonly used technique for empirical calibration in the COCOMO Ⅱ model. But it has a few assumptions frequently violated by software engineering data sets. It is true that the source data is also generally imprecise in reporting size, effort, and cost-driver ratings, particularly across different organizations. And that the outlier for the source data is a peculiarity and indicates a data pint To cope with difficulties, in this paper, we propose a new regression method for calibrating COCOMO Ⅱ post-architecture model based on the minimum relative erro(MRE) criterion. The characteristic of the proposed method is insensitive to the extreme values of the data in the empirical calibration. As the experimental results, It is evident that our proposed calibration method MRE was shown to be superior to the traditional regression approach for model calibration, as illustrated by the values obtained for standard deviation(^σ), and prediction at level L PRED(L) measures.

  • PDF

Statistical Analysis for Chemical Characterization of Fall-Out Particles (강하분진의 화학적 특성파악을 위한 통계학적 해석)

  • Kim, Hyeon-Seop;Heo, Jeong-Suk;Kim, Dong-Sul
    • Journal of Korean Society for Atmospheric Environment
    • /
    • v.14 no.6
    • /
    • pp.631-642
    • /
    • 1998
  • Fall-out particles were collected by the modified British deposit gauges at 35 sampling sites in Suwon area from January to November, 1996. Twenty chemical species (Al. Ba, Cd, Cr, K, Pb, Sb, Zn, Cu, Fe, Ni, V, F-, Cl-, NO3-, 5042-, Na+, NH4+, Mg2+, and Ca2+) were analyzed by AAS and If. The purposes of this study were to estimate qualitatively various emission sources of the fell-out particle by applying multivariate statistical techniques such as factor analysis, multiple regression analysis, and discriminant analysis. During the study, outlier sites were determined by a z-score method. Cl-, Na+, Mg2+, and SO42- were highly correlated due to their common marine related source. Wind speed was the most influential factor for the deposition fluxes of the particle itself and all the chemical species as well. When applying the factor analysis, 8 source patterns were qualitatively obtained, such as marine source, soil source, oil burning source, Cr related source, tire source, Cd related source, agriculture source, and F- related source. As a result of the multiple regression analysis, we could suggest that some chemical compounds may possibly exist in the form of CaSO4, NaN03, NaCl, MgC12, (NH4)2SO4, NaF, and CaCl2 in the fall-out particles. Finally, spatial and seasonal classification study performed by a discriminant analysis showed th.at SO42-, Ca2+, Cl-, and Fe were dominant in the group of spatial pattern; however, SO42-, Cl-, Al, and V were in the group of seasonal pattern.

  • PDF

An Improved Calibration Method for the COCOMO II Post-Architecture Model

  • Yoon, Myoung-Young
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.5 no.2
    • /
    • pp.47-55
    • /
    • 2000
  • To date many software engineering cost models have been developed to predict cost, schedule, and effort of the software under development. The COCOMO Ⅱ is well- suited for the new software development life cycle such as non-sequential and rapid- development processes. The traditional regression approach based on the least square criterion is the most commonly used technique for empirical calibration in the COCOMO Ⅱ model. It has a few assumptions frequently violated by software engineering data sets. The source data is also generally imprecise in reporting size effort, and cost-driver ratings, particularly across different organizations. And that the outlier for the source data is a peculiarity and indicates a data point. To cope with difficulties, in this paper, we propose a new regression method for calibrating COCOMO Ⅱ post-architecture model based on the minimum relative error(MRE) criterion. The characteristic of the proposed method is insensitive to the extreme values of the data in the empirical calibration. As the experimental results, It is evident that our proposed calibration method MRE was shown to be superior to the traditional regression approach for model calibration, as illustrated by the values obtained for standard deviation(^σ), and prediction at level LPRED(L) measures.

  • PDF

An Objective No-Reference Perceptual Quality Assessment Metric based on Temporal Complexity and Disparity for Stereoscopic Video

  • Ha, Kwangsung;Bae, Sung-Ho;Kim, Munchurl
    • IEIE Transactions on Smart Processing and Computing
    • /
    • v.2 no.5
    • /
    • pp.255-265
    • /
    • 2013
  • 3DTV is expected to be a promising next-generation broadcasting service. On the other hand, the visual discomfort/fatigue problems caused by viewing 3D videos have become an important issue. This paper proposes a perceptual quality assessment metric for a stereoscopic video (SV-PQAM). To model the SV-PQAM, this paper presents the following features: temporal variance, disparity variation in intra-frames, disparity variation in inter-frames and disparity distribution of frame boundary areas, which affect the human perception of depth and visual discomfort for stereoscopic views. The four features were combined into the SV-PQAM, which then becomes a no-reference stereoscopic video quality perception model, as an objective quality assessment metric. The proposed SV-PQAM does not require a depth map but instead uses the disparity information by a simple estimation. The model parameters were estimated based on linear regression from the mean score opinion values obtained from the subjective perception quality assessments. The experimental results showed that the proposed SV-PQAM exhibits high consistency with subjective perception quality assessment results in terms of the Pearson correlation coefficient value of 0.808, and the prediction performance exhibited good consistency with a zero outlier ratio value.

  • PDF

Modified BLS Weight Adjustment (수정된 BLS 가중치보정법)

  • Park, Jung-Joon;Cho, Ki-Jong;Lee, Sang-Eun;Shin, Key-Il
    • Communications for Statistical Applications and Methods
    • /
    • v.18 no.3
    • /
    • pp.367-376
    • /
    • 2011
  • BLS weight adjustment is a widely used method for business surveys with non-responses and outliers. Recent surveys show that the non-response weight adjustment of the BLS method is the same as the ratio imputation method. In this paper, we suggested a modified BLS weight adjustment method by imputing missing values instead of using weight adjustment for non-response. Monthly labor survey data is used for a small Monte-Carlo simulation and we conclude that the suggested method is superior to the original BLS weight adjustment method.

Regression diagnostics for response transformations in a partial linear model (부분선형모형에서 반응변수변환을 위한 회귀진단)

  • Seo, Han Son;Yoon, Min
    • Journal of the Korean Data and Information Science Society
    • /
    • v.24 no.1
    • /
    • pp.33-39
    • /
    • 2013
  • In the transformation of response variable in partial linear models outliers can cause a bad effect on estimating the transformation parameter, just as in the linear models. To solve this problem the processes of estimating transformation parameter and detecting outliers are needed, but have difficulties to be performed due to the arbitrariness of the nonparametric function included in the partial linear model. In this study, through the estimation of nonparametric function and outlier detection methods such as a sequential test and a maximum trimmed likelihood estimation, processes for transforming response variable robust to outliers in partial linear models are suggested. The proposed methods are verified and compared their effectiveness by simulation study and examples.