• Title/Summary/Keyword: Time-series Data

Search Result 3,688, Processing Time 0.031 seconds

Efficient Compression Algorithm with Limited Resource for Continuous Surveillance

  • Yin, Ling;Liu, Chuanren;Lu, Xinjiang;Chen, Jiafeng;Liu, Caixing
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.11
    • /
    • pp.5476-5496
    • /
    • 2016
  • Energy efficiency of resource-constrained wireless sensor networks is critical in applications such as real-time monitoring/surveillance. To improve the energy efficiency and reduce the energy consumption, the time series data can be compressed before transmission. However, most of the compression algorithms for time series data were developed only for single variate scenarios, while in practice there are often multiple sensor nodes in one application and the collected data is actually multivariate time series. In this paper, we propose to compress the time series data by the Lasso (least absolute shrinkage and selection operator) approximation. We show that, our approach can be naturally extended for compressing the multivariate time series data. Our extension is novel since it constructs an optimal projection of the original multivariates where the best energy efficiency can be realized. The two algorithms are named by ULasso (Univariate Lasso) and MLasso (Multivariate Lasso), for which we also provide practical guidance for parameter selection. Finally, empirically evaluation is implemented with several publicly available real-world data sets from different application domains. We quantify the algorithm performance by measuring the approximation error, compression ratio, and computation complexity. The results show that ULasso and MLasso are superior to or at least equivalent to compression performance of LTC and PLAMlis. Particularly, MLasso can significantly reduce the smooth multivariate time series data, without breaking the major trends and important changes of the sensor network system.

Design of Multi-Level Abnormal Detection System Suitable for Time-Series Data (시계열 데이터에 적합한 다단계 비정상 탐지 시스템 설계)

  • Chae, Moon-Chang;Lim, Hyeok;Kang, Namhi
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.16 no.6
    • /
    • pp.1-7
    • /
    • 2016
  • As new information and communication technologies evolve, security threats are also becoming increasingly intelligent and advanced. In this paper, we analyze the time series data continuously entered through a series of periods from the network device or lightweight IoT (Internet of Things) devices by using the statistical technique and propose a system to detect abnormal behaviors of the device or abnormality based on the analysis results. The proposed system performs the first level abnormal detection by using previously entered data set, thereafter performs the second level anomaly detection according to the trust bound configured by using stored time series data based on time attribute or group attribute. Multi-level analysis is able to improve reliability and to reduce false positives as well through a variety of decision data set.

Design of Hierarchically Structured Clustering Algorithm and its Application (계층 구조 클러스터링 알고리즘 설계 및 그 응용)

  • Bang, Young-Keun;Park, Ha-Yong;Lee, Chul-Heui
    • Journal of Industrial Technology
    • /
    • v.29 no.B
    • /
    • pp.17-23
    • /
    • 2009
  • In many cases, clustering algorithms have been used for extracting and discovering useful information from non-linear data. They have made a great effect on performances of the systems dealing with non-linear data. Thus, this paper presents a new approach called hierarchically structured clustering algorithm, and it is applied to the prediction system for non-linear time series data. The proposed hierarchically structured clustering algorithm (called HCKA: Hierarchical Cross-correlation and K-means clustering Algorithms) in which the cross-correlation and k-means clustering algorithm are combined can accept the correlationship of non-linear time series as well as statistical characteristics. First, the optimal differences of data are generated, which can suitably reveal the characteristics of non-linear time series. Second, the generated differences are classified into the upper clusters for their predictors by the cross-correlation clustering algorithm, and then each classified differences are classified again into the lower fuzzy sets by the k-means clustering algorithm. As a result, the proposed method can give an efficient classification and improve the performance. Finally, we demonstrates the effectiveness of the proposed HCKA via typical time series examples.

  • PDF

MLOps workflow language and platform for time series data anomaly detection

  • Sohn, Jung-Mo;Kim, Su-Min
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.11
    • /
    • pp.19-27
    • /
    • 2022
  • In this study, we propose a language and platform to describe and manage the MLOps(Machine Learning Operations) workflow for time series data anomaly detection. Time series data is collected in many fields, such as IoT sensors, system performance indicators, and user access. In addition, it is used in many applications such as system monitoring and anomaly detection. In order to perform prediction and anomaly detection of time series data, the MLOps platform that can quickly and flexibly apply the analyzed model to the production environment is required. Thus, we developed Python-based AI/ML Modeling Language (AMML) to easily configure and execute MLOps workflows. Python is widely used in data analysis. The proposed MLOps platform can extract and preprocess time series data from various data sources (R-DB, NoSql DB, Log File, etc.) using AMML and predict it through a deep learning model. To verify the applicability of AMML, the workflow for generating a transformer oil temperature prediction deep learning model was configured with AMML and it was confirmed that the training was performed normally.

Carbonation depth prediction of concrete bridges based on long short-term memory

  • Youn Sang Cho;Man Sung Kang;Hyun Jun Jung;Yun-Kyu An
    • Smart Structures and Systems
    • /
    • v.33 no.5
    • /
    • pp.325-332
    • /
    • 2024
  • This study proposes a novel long short-term memory (LSTM)-based approach for predicting carbonation depth, with the aim of enhancing the durability evaluation of concrete structures. Conventional carbonation depth prediction relies on statistical methodologies using carbonation influencing factors and in-situ carbonation depth data. However, applying in-situ data for predictive modeling faces challenges due to the lack of time-series data. To address this limitation, an LSTM-based carbonation depth prediction technique is proposed. First, training data are generated through random sampling from the distribution of carbonation velocity coefficients, which are calculated from in-situ carbonation depth data. Subsequently, a Bayesian theorem is applied to tailor the training data for each target bridge, which are depending on surrounding environmental conditions. Ultimately, the LSTM model predicts the time-dependent carbonation depth data for the target bridge. To examine the feasibility of this technique, a carbonation depth dataset from 3,960 in-situ bridges was used for training, and untrained time-series data from the Miho River bridge in the Republic of Korea were used for experimental validation. The results of the experimental validation demonstrate a significant reduction in prediction error from 8.19% to 1.75% compared with the conventional statistical method. Furthermore, the LSTM prediction result can be enhanced by sequentially updating the LSTM model using actual time-series measurement data.

IGARCH and Stochastic Volatility : Case Study

  • Hwang, S.Y.;Park, J.A.
    • Journal of the Korean Data and Information Science Society
    • /
    • v.16 no.4
    • /
    • pp.835-841
    • /
    • 2005
  • IGARCH and Stochastic Volatility Model(SVM, for short) have frequently provided useful approximations to the real aspects of financial time series. This article is concerned with modeling various Korean financial time series using both IGARCH and stochastic volatility models. Daily data sets with sample period ranging from 2000 and 2004 including KOSPI, KOSDAQ and won-dollar exchange rate are comparatively analyzed using IGARCH and SVM.

  • PDF

A Simultaneous Test for Multivariate Normality and Independence with Application to Univariate Residuals

  • Park, Cheol-Yong
    • Journal of the Korean Data and Information Science Society
    • /
    • v.17 no.1
    • /
    • pp.115-122
    • /
    • 2006
  • A test is suggested for detecting deviations from both multivariate normality and independence. This test can be used for assessing the normality and independence of univariate time series residuals. We derive the limiting distribution of the test statistic and a simulation study is conducted to study the accuracy of the limiting distribution in finite samples. Finally, we apply our method to a real data of time series.

  • PDF

Bayesian Neural Network with Recurrent Architecture for Time Series Prediction

  • Hong, Chan-Young;Park, Jung-Hun;Yoon, Tae-Sung;Park, Jin-Bae
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2004.08a
    • /
    • pp.631-634
    • /
    • 2004
  • In this paper, the Bayesian recurrent neural network (BRNN) is proposed to predict time series data. Among the various traditional prediction methodologies, a neural network method is considered to be more effective in case of non-linear and non-stationary time series data. A neural network predictor requests proper learning strategy to adjust the network weights, and one need to prepare for non-linear and non-stationary evolution of network weights. The Bayesian neural network in this paper estimates not the single set of weights but the probability distributions of weights. In other words, we sets the weight vector as a state vector of state space method, and estimates its probability distributions in accordance with the Bayesian inference. This approach makes it possible to obtain more exact estimation of the weights. Moreover, in the aspect of network architecture, it is known that the recurrent feedback structure is superior to the feedforward structure for the problem of time series prediction. Therefore, the recurrent network with Bayesian inference, what we call BRNN, is expected to show higher performance than the normal neural network. To verify the performance of the proposed method, the time series data are numerically generated and a neural network predictor is applied on it. As a result, BRNN is proved to show better prediction result than common feedforward Bayesian neural network.

  • PDF

A model of predicting performance of Olympic female weightlifters using time series analysis

  • Won, Jin-hee;Cho, In-ho
    • International Journal of Advanced Culture Technology
    • /
    • v.8 no.3
    • /
    • pp.216-222
    • /
    • 2020
  • The purpose of this study was to predict the performance of female weightlifters using time series analysis. Based on this purpose, a time series analysis was used to calculate the performance prediction model for women(58kg) among the domestic women weightlifters who participated in the Olympics. As a result of creating time series data based on 10 years of record and then evaluating the sequential charts of each athlete group, the female athletes' records did not show any seasonality or difference. In addition, after examining the independence of the data through the creation of a time series model, it was shown that the models produced conformed to the criteria for compliance and that there was no difference in the data, but there was a trend. Accordingly, Holt linear trend analysis of the exponential smoothing model was applied. As a result of deriving the prediction model of the athletes through this process, it was found that the women (58kg) who participated in the Olympics continued to improve within the range of 166.11kg to 184.1kg.

Development of 3D Visualization Technology for Meteorological Data Using IDL (IDL을 이용한 기상자료 3 차원 가시화 기술개발 연구)

  • Joh Min-su;Yun Ja-Young;Seo In-Bum
    • 한국가시화정보학회:학술대회논문집
    • /
    • 2002.11a
    • /
    • pp.77-80
    • /
    • 2002
  • The recent 3D visualization such as volume rendering, iso-surface rendering or stream line visualization gives more understanding about structures or distribution of data in a space and, moreover, the real-time rendering of a scene enables the animation of time-series data. Because the meteorological data is frequently formed as multi-variables, 3-dimensional and time-series data, the spatial analysis, time-series analysis, vector display, and animation techniques can do important roles to get more understanding about data. In this research, our aim is to develop the 3-dimensional visualization techniques for meteorological data in the PC environment by using IDL. The visualization technology from :his research will be used as basic technology not only for the deeper understanding and the more exact prediction about meteorological environments but also for the scientific and spatial data visualization research in any field from which three-dimensional data comes out such as oceanography, earth science, or aeronautical engineering.

  • PDF