• Title/Summary/Keyword: TimeSeries Data

Search Result 3,692, Processing Time 0.03 seconds

A Pre-processing Process Using TadGAN-based Time-series Anomaly Detection (TadGAN 기반 시계열 이상 탐지를 활용한 전처리 프로세스 연구)

  • Lee, Seung Hoon;Kim, Yong Soo
    • Journal of Korean Society for Quality Management
    • /
    • v.50 no.3
    • /
    • pp.459-471
    • /
    • 2022
  • Purpose: The purpose of this study was to increase prediction accuracy for an anomaly interval identified using an artificial intelligence-based time series anomaly detection technique by establishing a pre-processing process. Methods: Significant variables were extracted by applying feature selection techniques, and anomalies were derived using the TadGAN time series anomaly detection algorithm. After applying machine learning and deep learning methodologies using normal section data (excluding anomaly sections), the explanatory power of the anomaly sections was demonstrated through performance comparison. Results: The results of the machine learning methodology, the performance was the best when SHAP and TadGAN were applied, and the results in the deep learning, the performance was excellent when Chi-square Test and TadGAN were applied. Comparing each performance with the papers applied with a Conventional methodology using the same data, it can be seen that the performance of the MLR was significantly improved to 15%, Random Forest to 24%, XGBoost to 30%, Lasso Regression to 73%, LSTM to 17% and GRU to 19%. Conclusion: Based on the proposed process, when detecting unsupervised learning anomalies of data that are not actually labeled in various fields such as cyber security, financial sector, behavior pattern field, SNS. It is expected to prove the accuracy and explanation of the anomaly detection section and improve the performance of the model.

Neural network AR model with ETS inputs (지수평활법을 외생변수로 사용하는 자기회귀 신경망 모형)

  • Minjae Kim;Byeongchan Seong
    • The Korean Journal of Applied Statistics
    • /
    • v.37 no.3
    • /
    • pp.297-309
    • /
    • 2024
  • This paper evaluates the performance of the neural network autoregressive model combined with an exponential smoothing model, called the NNARX+ETS model. The combined model utilizes the components of ETS as exogenous variables for NNARX, to forecast time series data using artificial neural networks. The main idea is to enhance the performance of NNAR using only lags of the original time series data, by combining traditional time series analysis methods with the neural networks through NNARX. We employ two real data for performance evaluation and compare the NNARX+ETS with NNAR and traditional time series analysis methods such as ETS and ARIMA (autoregressive integrated moving average) models.

Non-stationary statistical modeling of extreme wind speed series with exposure correction

  • Huang, Mingfeng;Li, Qiang;Xu, Haiwei;Lou, Wenjuan;Lin, Ning
    • Wind and Structures
    • /
    • v.26 no.3
    • /
    • pp.129-146
    • /
    • 2018
  • Extreme wind speed analysis has been carried out conventionally by assuming the extreme series data is stationary. However, time-varying trends of the extreme wind speed series could be detected at many surface meteorological stations in China. Two main reasons, exposure change and climate change, were provided to explain the temporal trends of daily maximum wind speed and annual maximum wind speed series data, recorded at Hangzhou (China) meteorological station. After making a correction on wind speed series for time varying exposure, it is necessary to perform non-stationary statistical modeling on the corrected extreme wind speed data series in addition to the classical extreme value analysis. The generalized extreme value (GEV) distribution with time-dependent location and scale parameters was selected as a non-stationary model to describe the corrected extreme wind speed series. The obtained non-stationary extreme value models were then used to estimate the non-stationary extreme wind speed quantiles with various mean recurrence intervals (MRIs) considering changing climate, and compared to the corresponding stationary ones with various MRIs for the Hangzhou area in China. The results indicate that the non-stationary property or dependence of extreme wind speed data should be carefully evaluated and reflected in the determination of design wind speeds.

Application of Multi-periodic Harmonic Model for Classification of Multi-temporal Satellite Data: MODIS and GOCI Imagery

  • Jung, Myunghee;Lee, Sang-Hoon
    • Korean Journal of Remote Sensing
    • /
    • v.35 no.4
    • /
    • pp.573-587
    • /
    • 2019
  • A multi-temporal approach using remotely sensed time series data obtained over multiple years is a very useful method for monitoring land covers and land-cover changes. While spectral-based methods at any particular time limits the application utility due to instability of the quality of data obtained at that time, the approach based on the temporal profile can produce more accurate results since data is analyzed from a long-term perspective rather than on one point in time. In this study, a multi-temporal approach applying a multi-periodic harmonic model is proposed for classification of remotely sensed data. A harmonic model characterizes the seasonal variation of a time series by four parameters: average level, frequency, phase, and amplitude. The availability of high-quality data is very important for multi-temporal analysis.An satellite image usually have many unobserved data and bad-quality data due to the influence of observation environment and sensing system, which impede the analysis and might possibly produce inaccurate results. Harmonic analysis is also very useful for real-time data reconstruction. Multi-periodic harmonic model is applied to the reconstructed data to classify land covers and monitor land-cover change by tracking the temporal profiles. The proposed method is tested with the MODIS and GOCI NDVI time series over the Korean Peninsula for 5 years from 2012 to 2016. The results show that the multi-periodic harmonic model has a great potential for classification of land-cover types and monitoring of land-cover changes through characterizing annual temporal dynamics.

Condition assessment of stay cables through enhanced time series classification using a deep learning approach

  • Zhang, Zhiming;Yan, Jin;Li, Liangding;Pan, Hong;Dong, Chuanzhi
    • Smart Structures and Systems
    • /
    • v.29 no.1
    • /
    • pp.105-116
    • /
    • 2022
  • Stay cables play an essential role in cable-stayed bridges. Severe vibrations and/or harsh environment may result in cable failures. Therefore, an efficient structural health monitoring (SHM) solution for cable damage detection is necessary. This study proposes a data-driven method for immediately detecting cable damage from measured cable forces by recognizing pattern transition from the intact condition when damage occurs. In the proposed method, pattern recognition for cable damage detection is realized by time series classification (TSC) using a deep learning (DL) model, namely, the long short term memory fully convolutional network (LSTM-FCN). First, a TSC classifier is trained and validated using the cable forces (or cable force ratios) collected from intact stay cables, setting the segmented data series as input and the cable (or cable pair) ID as class labels. Subsequently, the classifier is tested using the data collected under possible damaged conditions. Finally, the cable or cable pair corresponding to the least classification accuracy is recommended as the most probable damaged cable or cable pair. A case study using measured cable forces from an in-service cable-stayed bridge shows that the cable with damage can be correctly identified using the proposed DL-TSC method. Compared with existing cable damage detection methods in the literature, the DL-TSC method requires minor data preprocessing and feature engineering and thus enables fast and convenient early detection in real applications.

Performance Analysis of the Time-series Pattern Index File for Content-based Music Genre Retrieval (내용기반 음악장르 검색에서 시계열 패턴 인덱스 화일의 성능 분석)

  • Kim, Young-In;Kim, Seon-Jong
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.11 no.5
    • /
    • pp.18-27
    • /
    • 2006
  • Rapid increase of the amount of music data demands for a new method that allows efficient similarity retrieval of music genre using audio features in music databases. To build this similarity retrieval, an indexing techniques that support audio features as a time-series pattern and data mining technologies are needed. In this paper, we address the development of a system that retrieves similar genre music based on the indexing techniques. We first propose the structure of content-based music genre retrieval system based on the time-series pattern index file and data mining technologies. In addition, we implement the time-series pattern index file using audio features and present performance analysis of the time-series pattern index file for similar genre retrieval. The experiments are performed on real data to verify the performance of the proposed method.

  • PDF

A Study on Time Series Cross-Validation Techniques for Enhancing the Accuracy of Reservoir Water Level Prediction Using Automated Machine Learning TPOT (자동기계학습 TPOT 기반 저수위 예측 정확도 향상을 위한 시계열 교차검증 기법 연구)

  • Bae, Joo-Hyun;Park, Woon-Ji;Lee, Seoro;Park, Tae-Seon;Park, Sang-Bin;Kim, Jonggun;Lim, Kyoung-Jae
    • Journal of The Korean Society of Agricultural Engineers
    • /
    • v.66 no.1
    • /
    • pp.1-13
    • /
    • 2024
  • This study assessed the efficacy of improving the accuracy of reservoir water level prediction models by employing automated machine learning models and efficient cross-validation methods for time-series data. Considering the inherent complexity and non-linearity of time-series data related to reservoir water levels, we proposed an optimized approach for model selection and training. The performance of twelve models was evaluated for the Obong Reservoir in Gangneung, Gangwon Province, using the TPOT (Tree-based Pipeline Optimization Tool) and four cross-validation methods, which led to the determination of the optimal pipeline model. The pipeline model consisting of Extra Tree, Stacking Ridge Regression, and Simple Ridge Regression showed outstanding predictive performance for both training and test data, with an R2 (Coefficient of determination) and NSE (Nash-Sutcliffe Efficiency) exceeding 0.93. On the other hand, for predictions of water levels 12 hours later, the pipeline model selected through time-series split cross-validation accurately captured the change pattern of time-series water level data during the test period, with an NSE exceeding 0.99. The methodology proposed in this study is expected to greatly contribute to the efficient generation of reservoir water level predictions in regions with high rainfall variability.

Hydrologic Modeling Approach using Time-Lag Recurrent Neural Networks Model (시간지체 순환신경망모형을 이용한 수문학적 모형화기법)

  • Kim, Seong-Won
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2010.05a
    • /
    • pp.1439-1442
    • /
    • 2010
  • Time-lag recurrent neural networks model (Time-Lag RNNM) is used to estimate daily pan evaporation (PE) using limited climatic variables such as max temperature ($T_{max}$), min temperature ($T_{min}$), mean wind speed ($W_{mean}$) and mean relative humidity ($RH_{mean}$). And, for the performances of Time-Lag RNNM, it is composed of training and test performances, respectively. The training and test performances are carried out using daily time series data, respectively. From this research, we evaluate the impact of Time-Lag RNNM for the modeling of the nonlinear time series data. We should, thus, construct the credible data of the daily PE using Time-Lag RNNM, and can suggest the methodology for the irrigation and drainage networks system. Furthermore, this research represents that the strong nonlinear relationship such as pan evaporation modeling can be generalized using Time-Lag RNNM.

  • PDF

Trend and Shift Analysis for Hydrologic and Climate Series (수문 및 기후 자료에 대한 선형 경향성 및 평균이동 분석)

  • Oh, Je Seung;Kim, Hung Soo;Seo, Byung Ha
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.26 no.4B
    • /
    • pp.355-362
    • /
    • 2006
  • Several techniques of MK test, Spearman's Rho test, Linear Regression test, CUSUM test, Cumulative Deviation, Worsley Likelihood Ratio test, Rank Sum test, and Students' t test were applied to detect the trends of slope and shift which exist in hydrologic and climate time series. The time series of annual rainfall, inflow, tree ring index, and southern oscillation index (SOI) were used and the trends of these series were compared in the study. From the results, it can be found that the data could be classified into two categories such as linear trend and shift. 4 series data of 8 rainfall series which reveal the trend show the shift and 8 series data of 18 tree ring index and March and April series of monthly SOI data show shift. Moreover, ADF test and BDS test were used to test stationarity and non-linearity of the data. In conclusion, through the study, various trend analysis techniques were compared and 6 kinds of characteristics which can exist in hydrologic time series were identified.

Prediction for Nonlinear Time Series Data using Neural Network (신경망을 이용한 비선형 시계열 자료의 예측)

  • Kim, Inkyu
    • Journal of Digital Convergence
    • /
    • v.10 no.9
    • /
    • pp.357-362
    • /
    • 2012
  • We have compared and predicted for non-linear time series data which are real data having different variences using GRCA(1) model and neural network method. In particular, using Korea Composite Stock Price Index rate, mean square errors of prediction are obtained in genaralized random coefficient autoregressive model and neural network method. Neural network method prove to be better in short-term forecasting, however GRCA(1) model perform well in long-term forecasting.