• Title/Summary/Keyword: Outlier Removal

Search Result 57, Processing Time 0.033 seconds

The Consideration on Calculation of Optimal Travel Speeds based on Analysis of AVI Data (AVI 수집 자료 분석에 근거한 최적 통행속도 산출에 관한 고찰)

  • Jeong, Yeon Tak;Jung, Hun Young
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.35 no.3
    • /
    • pp.625-637
    • /
    • 2015
  • This study aims to calculate optimal travel speeds based on analysis of the AVI data collected in the uninterrupted traffic flow, and the results are as follows. Firstly, we looked into the distribution of the sectional travel times of each probe vehicle and compared the difference in the sectional travel speeds of each probe vehicle. As a result, it is shown that outliers should be removed for the distribution of the sectional travel times. Secondly, there were differences among type 1(passenger automobiles) & type 2(automobiles for passengers and freight) and type 4(special automobiles) in the non-congestion section. thus it was revealed that there is a necessity to remove type 4(special automobiles) when calculating the sectional travel speeds. Thirdly, Based on the results of these, the optimal outlier removal procedures were applied to this study. As a result, it showed that the MAPE was between 0.3% and 2.0% and RMSE was between 0.3 and 2.3 which are very similar figures to the actual average traffic speed. Also, the minimum sample size was satisfied at the confidence level of 95%. The result of study is expected to serve as a useful basis for the local government to build the AVI. In the future, it will be necessary to study to integrate AVI data and other data for more accurate traffic information.

Prediction of Uniaxial Compressive Strength of Rock using Shield TBM Machine Data and Machine Learning Technique (쉴드 TBM 기계 데이터 및 머신러닝 기법을 이용한 암석의 일축압축강도 예측)

  • Kim, Tae-Hwan;Ko, Tae Young;Park, Yang Soo;Kim, Taek Kon;Lee, Dae Hyuk
    • Tunnel and Underground Space
    • /
    • v.30 no.3
    • /
    • pp.214-225
    • /
    • 2020
  • Uniaxial compressive strength (UCS) of rock is one of the important factors to determine the advance speed during shield TBM tunnel excavation. UCS can be obtained through the Geotechnical Data Report (GDR), and it is difficult to measure UCS for all tunneling alignment. Therefore, the purpose of this study is to predict UCS by utilizing TBM machine driving data and machine learning technique. Several machine learning techniques were compared to predict UCS, and it was confirmed the stacking model has the most successful prediction performance. TBM machine data and UCS used in the analysis were obtained from the excavation of rock strata with slurry shield TBMs. The data were divided into 8:2 for training and test and pre-processed including feature selection, scaling, and outlier removal. After completing the hyper-parameter tuning, the stacking model was evaluated with the root-mean-square error (RMSE) and the determination coefficient (R2), and it was found to be 5.556 and 0.943, respectively. Based on the results, the sacking models are considered useful in predicting rock strength with TBM excavation data.

A Study on Calculation of Sectional Travel Speeds of the Interrupted Traffic Flow with the Consideration of the Characteristics of Probe Data (프로브 자료의 특성을 고려한 단속류의 구간 통행속도 산출에 관한 연구)

  • Jeong, Yeon Tak;Jung, Hun Young
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.34 no.6
    • /
    • pp.1851-1861
    • /
    • 2014
  • This study aims to calculate reliable sectional travel speeds with the consideration of the characteristics of the probe data collected in the interrupted traffic flow. First, in order to analysis the characteristics of the probe data, we looked into the distribution of the sectional travel times of each probe vehicle and compared the difference in the sectional travel speeds of each probe vehicle collected by DSRC. As a result, it is shown that outliers should be removed for the distribution of the sectional travel times. However, The comparison results show that the sectional travel speeds from the DSRC probe vehicles are not significantly different. Finally, based on the distribution characteristics of the sectional travel speeds of each probe vehicle and the representative values counted during a collection period, we drew the optimal outlier removal procedure and evaluated the estimation errors. The evaluation results showed that the DSRC sectional travel speeds were found to be similar to the observed values from actually running vehicles. On the contrary, in the case of the sectional travel speeds of intra-city bus, it was analyzed that they were less accurate than the DSRC sectional travel speeds. In the future, it will be necessary to improve BIS process and make use of the travel information on intra-city buses collected in real time to find various ways of applying it as traffic information.

Speckle Noise Removal by Rank-ordered Differences Diffusion Filter (순위 차 확산 필터를 이용한 스페클 잡음 제거)

  • Ye, Chul-Soo
    • Korean Journal of Remote Sensing
    • /
    • v.25 no.1
    • /
    • pp.21-30
    • /
    • 2009
  • The purposes of this paper are to present a selection method of neighboring pixels whose local statistics are similar to the center pixel and combine the selection result with mean curvature diffusion filter to reduce noises in remote sensed imagery. The order of selection of neighboring pixels is critical, especially for finding a pixel belonging to the homogeneous region, since the statistics of the homogeneous region vary according to the selection order. An effective strategy for selecting neighboring pixels, which uses rank-order differences vector obtained by computing the intensity differences between the center pixel and neighboring pixels and arranging them in ascending order, is proposed in this paper. By using region growing method, we divide the elements of the rank-ordered differences vector into two groups, homogeneous rank-ordered differences vector and outlier rank-ordered differences vector. The mean curvature diffusion filter is combined with a line process, which chooses selectively diffusion coefficient of the neighboring pixels belonging into homogeneous rank-ordered differences vector. Experimental results using an aerial image and a TerraSAR-X satellite image showed that the proposed method reduced more efficiently noises than some conventional adaptive filters using all neighboring pixels in updating the center pixel.

Reconfiguration of Physical Structure of Vegetation by Voxelization Based on 3D Point Clouds (3차원 포인트 클라우드 기반 복셀화에 의한 식생의 물리적 구조 재구현)

  • Ahn, Myeonghui;Jang, Eun-kyung;Bae, Inhyeok;Ji, Un
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.40 no.6
    • /
    • pp.571-581
    • /
    • 2020
  • Vegetation affects water level change and flow resistance in rivers and impacts waterway ecosystems as a whole. Therefore, it is important to have accurate information about the species, shape, and size of any river vegetation. However, it is not easy to collect full vegetation data on-site, so recent studies have attempted to obtain large amounts of vegetation data using terrestrial laser scanning (TLS). Also, due to the complex shape of vegetation, it is not easy to obtain accurate information about the canopy area, and there are limitations due to a complex range of variables. Therefore, the physical structure of vegetation was analyzed in this study by reconfiguring high-resolution point cloud data collected through 3-dimensional terrestrial laser scanning (3D TLS) in a voxel. Each physical structure was analyzed under three different conditions: a simple vegetation formation without leaves, a complete formation with leaves, and a patch-scale vegetation formation. In the raw data, the outlier and unnecessary data were filtered and removed by Statistical Outlier Removal (SOR), resulting in 17%, 26%, and 25% of data being removed, respectively. Also, vegetation volume by voxel size was reconfigured from post-processed point clouds and compared with vegetation volume; the analysis showed that the margin of error was 8%, 25%, and 63% for each condition, respectively. The larger the size of the target sample, the larger the error. The vegetation surface looked visually similar when resizing the voxel; however, the volume of the entire vegetation was susceptible to error.

Pre-processing of load data of agricultural tractors during major field operations

  • Ryu, Myong-Jin;Kabir, Md. Shaha Nur;Choo, Youn-Kug;Chung, Sun-Ok;Kim, Yong-Joo;Ha, Jong-Kyou;Lee, Kyeong-Hwan
    • Korean Journal of Agricultural Science
    • /
    • v.42 no.1
    • /
    • pp.53-61
    • /
    • 2015
  • Development of highly efficient and energy-saving tractors has been one of the issues in agricultural machinery. For design of such tractors, measurement and analysis of load on major power transmission parts of the tractors are the most important pre-requisite tasks. Objective of this study was to perform pre-processing procedures before effective analysis of load data of agricultural tractors (30, 75, and 82 kW) during major field operations such as plow tillage, rotary tillage, baling, bale wrapping, and to select the suitable pre-processing method for the analysis. A load measurement systems, equipped in the tractors, were consisted of strain-gauge, encoder, hydraulic pressure, and radar speed sensors to measure torque and rotational speed levels of transmission input shaft, PTO shaft, and driving axle shafts, pressure of the hydraulic inlet line, and travel speed, respectively. The entire sensor data were collected at a 200-Hz rate. Plow tillage, rotary tillage, baling, wrapping, and loader operations were selected as major field operations of agricultural tractors. Same or different farm works and driving levels were set differently for each of the load measuring experiment. Before load data analysis, pre-processing procedures such as outlier removal, low-pass filtering, and data division were performed. Data beyond the scope of the measuring range of the sensors and the operating range of the power transmission parts were removed. Considering engine and PTO rotational speeds, frequency components greater than 90, 60, and 60 Hz cut off frequencies were low-pass filtered for plow tillage, rotary tillage, and baler operations, respectively. Measured load data were divided into five parts: driving, working, implement up, implement down, and turning. Results of the study would provide useful information for load characteristics of tractors on major field operations.

Pedestrian Counting System based on Average Filter Tracking for Measuring Advertisement Effectiveness of Digital Signage (디지털 사이니지의 광고효과 측정을 위한 평균 필터 추적 기반 유동인구 수 측정 시스템)

  • Kim, Kiyong;Yoon, Kyoungro
    • Journal of Broadcast Engineering
    • /
    • v.21 no.4
    • /
    • pp.493-505
    • /
    • 2016
  • Among modern computer vision and video surveillance systems, the pedestrian counting system is a one of important systems in terms of security, scheduling and advertising. In the field of, pedestrian counting remains a variety of challenges such as changes in illumination, partial occlusion, overlap and people detection. During pedestrian counting process, the biggest problem is occlusion effect in crowded environment. Occlusion and overlap must be resolved for accurate people counting. In this paper, we propose a novel pedestrian counting system which improves existing pedestrian tracking method. Unlike existing pedestrian tracking method, proposed method shows that average filter tracking method can improve tracking performance. Also proposed method improves tracking performance through frame compensation and outlier removal. At the same time, we keep various information of tracking objects. The proposed method improves counting accuracy and reduces error rate about S6 dataset and S7 dataset. Also our system provides real time detection at the rate of 80 fps.

Calibration Update for the Measuring Total Nitrogen Content in Rice Plant Tissue Using the Near Infrared Spectroscopy

  • Kwon, Young-Rip;Song, Young-Eun;Choi, Dong-Chil;Ryu, Jeong
    • KOREAN JOURNAL OF CROP SCIENCE
    • /
    • v.54 no.1
    • /
    • pp.29-35
    • /
    • 2009
  • The aim of the present study was to update the calibration that is used for the measurement of the total nitrogen content in the rice plant samples by using the visible and near infrared spectrum. Before the equation merge, correlation coefficient of calibration equation for nitrogen content on each rice parts was 0.945 (Leaf), 0.928 (Stem), and 0.864 (Whole plant), respectively. In the calibration models created by each part in the rice plant under the various regression method, the calibration model for the leaf was recorded with relatively high accuracy. Among of those, the calibration equation developed by Partial least squares (PLS) method was more accurate than the Multiple linear regression (MLR) method. The calibration equation was sensitive based on variety and location variations. However, we have merged and enlarged various of the samples that made not only to measure the nitrogen content more accurately, but also later sampling populations became more diversified. After merging, $R^2$ value becomes more accurate and significantly to 0.950 (L.), 0.974 (S.), 0.940 (W.). Also, after removal of outlier, R2 values increased into 0.998, 0.995, and 0.997. In view of the results so far achieved, Standard error of prediction (SEP) and SEP (C) were reduced in the stem and whole plant. Biases were reduced in the leaf, stem as well as whole plant. Slopes were high in the stem. Standard deviation reduced in the stem but $R^2$ was high in the stem and whole plant. Result was indicated that calibration equation make update, and updating robust calibration equation from merge function and multi-variate calibration.

An LSTM Neural Network Model for Forecasting Daily Peak Electric Load of EV Charging Stations (EV 충전소의 일별 최대전력부하 예측을 위한 LSTM 신경망 모델)

  • Lee, Haesung;Lee, Byungsung;Ahn, Hyun
    • Journal of Internet Computing and Services
    • /
    • v.21 no.5
    • /
    • pp.119-127
    • /
    • 2020
  • As the electric vehicle (EV) market in South Korea grows, it is required to expand charging facilities to respond to rapidly increasing EV charging demand. In order to conduct a comprehensive facility planning, it is necessary to forecast future demand for electricity and systematically analyze the impact on the load capacity of facilities based on this. In this paper, we design and develop a Long Short-Term Memory (LSTM) neural network model that predicts the daily peak electric load at each charging station using the EV charging data of KEPCO. First, we obtain refined data through data preprocessing and outlier removal. Next, our model is trained by extracting daily features per charging station and constructing a training set. Finally, our model is verified through performance analysis using a test set for each charging station type, and the limitations of our model are discussed.

Automatic generation of reliable DEM using DTED level 2 data from high resolution satellite images (고해상도 위성영상과 기존 수치표고모델을 이용하여 신뢰성이 향상된 수치표고모델의 자동 생성)

  • Lee, Tae-Yoon;Jung, Jae-Hoon;Kim, Tae-Jung
    • Spatial Information Research
    • /
    • v.16 no.2
    • /
    • pp.193-206
    • /
    • 2008
  • If stereo images is used for Digital Elevation Model (DEM) generation, a DEM is generally made by matching left image against right image from stereo images. In stereo matching, tie-points are used as initial match candidate points. The number and distribution of tie-points influence the matching result. DEM made from matching result has errors such as holes, peaks, etc. These errors are usually interpolated by neighbored pixel values. In this paper, we propose the DEM generation method combined with automatic tie-points extraction using existing DEM, image pyramid, and interpolating new DEM using existing DEM for more reliable DEM. For test, we used IKONOS, QuickBird, SPOT5 stereo images and a DTED level 2 data. The test results show that the proposed method automatically makes reliable DEMs. For DEM validation, we compared heights of DEM by proposed method with height of existing DTED level 2 data. In comparison result, RMSE was under than 15 m.

  • PDF