• Title/Summary/Keyword: time series prediction

Search Result 884, Processing Time 0.031 seconds

Analysis of Relative Settlement Behavior of Retaining Wall Backside Ground Using Clustering (군집분류를 이용한 흙막이 벽체 배면 지반의 상대적 침하거동 분석)

  • Young-Jun Kwack;Heui-Soo Han
    • The Journal of Engineering Geology
    • /
    • v.33 no.1
    • /
    • pp.189-200
    • /
    • 2023
  • As urbanization and industrialization increase development in downtown areas, damage due to ground settlement continues to occur. Building collapse in urban has a high risk of leading to large-scale damage to life and property. However, there has rarely been studied on measurement data analysis methods when uneven loads are applied to the excavated ground and no prior knowledge of the ground. Accordingly, it was attempted to analyze the relative settlement behavior and correlation by processing the time-series surface settlement of construction sites in the urban. In this paper, the average index of difference in settlement and average of relative difference in settlement are defined and calculated, then plotted in the coordinate system to analyze the relative settlement behavior over time. In addition, since there was no prior knowledge of the ground, a standard to classify the clusters was needed, and the observation points were classified into using k-means clustering and Dunn Index. As a result of the analysis, it was confirmed that all the clusters moved to the stable region as the settlement amount converges. The clusters were segmented. Based on the analysis results, it was possible to distinguish between the independent displacement area and same behavior area by analyzing the correlation between measurement points. If possible to analyze the relative settlement behavior between the stations and classify the behavior areas, it can be helpful in settlement and stability management, such as uplift of the surrounding area, prediction of ground failure area, and prevention of activity failure.

Multidimensional data generation of water distribution systems using adversarially trained autoencoder (적대적 학습 기반 오토인코더(ATAE)를 이용한 다차원 상수도관망 데이터 생성)

  • Kim, Sehyeong;Jun, Sanghoon;Jung, Donghwi
    • Journal of Korea Water Resources Association
    • /
    • v.56 no.7
    • /
    • pp.439-449
    • /
    • 2023
  • Recent advancements in data measuring technology have facilitated the installation of various sensors, such as pressure meters and flow meters, to effectively assess the real-time conditions of water distribution systems (WDSs). However, as cities expand extensively, the factors that impact the reliability of measurements have become increasingly diverse. In particular, demand data, one of the most significant hydraulic variable in WDS, is challenging to be measured directly and is prone to missing values, making the development of accurate data generation models more important. Therefore, this paper proposes an adversarially trained autoencoder (ATAE) model based on generative deep learning techniques to accurately estimate demand data in WDSs. The proposed model utilizes two neural networks: a generative network and a discriminative network. The generative network generates demand data using the information provided from the measured pressure data, while the discriminative network evaluates the generated demand outputs and provides feedback to the generator to learn the distinctive features of the data. To validate its performance, the ATAE model is applied to a real distribution system in Austin, Texas, USA. The study analyzes the impact of data uncertainty by calculating the accuracy of ATAE's prediction results for varying levels of uncertainty in the demand and the pressure time series data. Additionally, the model's performance is evaluated by comparing the results for different data collection periods (low, average, and high demand hours) to assess its ability to generate demand data based on water consumption levels.

A Study on Commodity Asset Investment Model Based on Machine Learning Technique (기계학습을 활용한 상품자산 투자모델에 관한 연구)

  • Song, Jin Ho;Choi, Heung Sik;Kim, Sun Woong
    • Journal of Intelligence and Information Systems
    • /
    • v.23 no.4
    • /
    • pp.127-146
    • /
    • 2017
  • Services using artificial intelligence have begun to emerge in daily life. Artificial intelligence is applied to products in consumer electronics and communications such as artificial intelligence refrigerators and speakers. In the financial sector, using Kensho's artificial intelligence technology, the process of the stock trading system in Goldman Sachs was improved. For example, two stock traders could handle the work of 600 stock traders and the analytical work for 15 people for 4weeks could be processed in 5 minutes. Especially, big data analysis through machine learning among artificial intelligence fields is actively applied throughout the financial industry. The stock market analysis and investment modeling through machine learning theory are also actively studied. The limits of linearity problem existing in financial time series studies are overcome by using machine learning theory such as artificial intelligence prediction model. The study of quantitative financial data based on the past stock market-related numerical data is widely performed using artificial intelligence to forecast future movements of stock price or indices. Various other studies have been conducted to predict the future direction of the market or the stock price of companies by learning based on a large amount of text data such as various news and comments related to the stock market. Investing on commodity asset, one of alternative assets, is usually used for enhancing the stability and safety of traditional stock and bond asset portfolio. There are relatively few researches on the investment model about commodity asset than mainstream assets like equity and bond. Recently machine learning techniques are widely applied on financial world, especially on stock and bond investment model and it makes better trading model on this field and makes the change on the whole financial area. In this study we made investment model using Support Vector Machine among the machine learning models. There are some researches on commodity asset focusing on the price prediction of the specific commodity but it is hard to find the researches about investment model of commodity as asset allocation using machine learning model. We propose a method of forecasting four major commodity indices, portfolio made of commodity futures, and individual commodity futures, using SVM model. The four major commodity indices are Goldman Sachs Commodity Index(GSCI), Dow Jones UBS Commodity Index(DJUI), Thomson Reuters/Core Commodity CRB Index(TRCI), and Rogers International Commodity Index(RI). We selected each two individual futures among three sectors as energy, agriculture, and metals that are actively traded on CME market and have enough liquidity. They are Crude Oil, Natural Gas, Corn, Wheat, Gold and Silver Futures. We made the equally weighted portfolio with six commodity futures for comparing with other commodity indices. We set the 19 macroeconomic indicators including stock market indices, exports & imports trade data, labor market data, and composite leading indicators as the input data of the model because commodity asset is very closely related with the macroeconomic activities. They are 14 US economic indicators, two Chinese economic indicators and two Korean economic indicators. Data period is from January 1990 to May 2017. We set the former 195 monthly data as training data and the latter 125 monthly data as test data. In this study, we verified that the performance of the equally weighted commodity futures portfolio rebalanced by the SVM model is better than that of other commodity indices. The prediction accuracy of the model for the commodity indices does not exceed 50% regardless of the SVM kernel function. On the other hand, the prediction accuracy of equally weighted commodity futures portfolio is 53%. The prediction accuracy of the individual commodity futures model is better than that of commodity indices model especially in agriculture and metal sectors. The individual commodity futures portfolio excluding the energy sector has outperformed the three sectors covered by individual commodity futures portfolio. In order to verify the validity of the model, it is judged that the analysis results should be similar despite variations in data period. So we also examined the odd numbered year data as training data and the even numbered year data as test data and we confirmed that the analysis results are similar. As a result, when we allocate commodity assets to traditional portfolio composed of stock, bond, and cash, we can get more effective investment performance not by investing commodity indices but by investing commodity futures. Especially we can get better performance by rebalanced commodity futures portfolio designed by SVM model.

Optimization of Support Vector Machines for Financial Forecasting (재무예측을 위한 Support Vector Machine의 최적화)

  • Kim, Kyoung-Jae;Ahn, Hyun-Chul
    • Journal of Intelligence and Information Systems
    • /
    • v.17 no.4
    • /
    • pp.241-254
    • /
    • 2011
  • Financial time-series forecasting is one of the most important issues because it is essential for the risk management of financial institutions. Therefore, researchers have tried to forecast financial time-series using various data mining techniques such as regression, artificial neural networks, decision trees, k-nearest neighbor etc. Recently, support vector machines (SVMs) are popularly applied to this research area because they have advantages that they don't require huge training data and have low possibility of overfitting. However, a user must determine several design factors by heuristics in order to use SVM. For example, the selection of appropriate kernel function and its parameters and proper feature subset selection are major design factors of SVM. Other than these factors, the proper selection of instance subset may also improve the forecasting performance of SVM by eliminating irrelevant and distorting training instances. Nonetheless, there have been few studies that have applied instance selection to SVM, especially in the domain of stock market prediction. Instance selection tries to choose proper instance subsets from original training data. It may be considered as a method of knowledge refinement and it maintains the instance-base. This study proposes the novel instance selection algorithm for SVMs. The proposed technique in this study uses genetic algorithm (GA) to optimize instance selection process with parameter optimization simultaneously. We call the model as ISVM (SVM with Instance selection) in this study. Experiments on stock market data are implemented using ISVM. In this study, the GA searches for optimal or near-optimal values of kernel parameters and relevant instances for SVMs. This study needs two sets of parameters in chromosomes in GA setting : The codes for kernel parameters and for instance selection. For the controlling parameters of the GA search, the population size is set at 50 organisms and the value of the crossover rate is set at 0.7 while the mutation rate is 0.1. As the stopping condition, 50 generations are permitted. The application data used in this study consists of technical indicators and the direction of change in the daily Korea stock price index (KOSPI). The total number of samples is 2218 trading days. We separate the whole data into three subsets as training, test, hold-out data set. The number of data in each subset is 1056, 581, 581 respectively. This study compares ISVM to several comparative models including logistic regression (logit), backpropagation neural networks (ANN), nearest neighbor (1-NN), conventional SVM (SVM) and SVM with the optimized parameters (PSVM). In especial, PSVM uses optimized kernel parameters by the genetic algorithm. The experimental results show that ISVM outperforms 1-NN by 15.32%, ANN by 6.89%, Logit and SVM by 5.34%, and PSVM by 4.82% for the holdout data. For ISVM, only 556 data from 1056 original training data are used to produce the result. In addition, the two-sample test for proportions is used to examine whether ISVM significantly outperforms other comparative models. The results indicate that ISVM outperforms ANN and 1-NN at the 1% statistical significance level. In addition, ISVM performs better than Logit, SVM and PSVM at the 5% statistical significance level.

Analysis of National Stream Drying Phenomena using DrySAT-WFT Model: Focusing on Inflow of Dam and Weir Watersheds in 5 River Basins (DrySAT-WFT 모형을 활용한 전국 하천건천화 분석: 전국 5대강 댐·보 유역의 유입량을 중심으로)

  • LEE, Yong-Gwan;JUNG, Chung-Gil;KIM, Won-Jin;KIM, Seong-Joon
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.23 no.2
    • /
    • pp.53-69
    • /
    • 2020
  • The increase of the impermeable area due to industrialization and urban development distorts the hydrological circulation system and cause serious stream drying phenomena. In order to manage this, it is necessary to develop a technology for impact assessment of stream drying phenomena, which enables quantitative evaluation and prediction. In this study, the cause of streamflow reduction was assessed for dam and weir watersheds in the five major river basins of South Korea by using distributed hydrological model DrySAT-WFT (Drying Stream Assessment Tool and Water Flow Tracking) and GIS time series data. For the modeling, the 5 influencing factors of stream drying phenomena (soil erosion, forest growth, road-river disconnection, groundwater use, urban development) were selected and prepared as GIS-based time series spatial data from 1976 to 2015. The DrySAT-WFT was calibrated and validated from 2005 to 2015 at 8 multipurpose dam watershed (Chungju, Soyang, Andong, Imha, Hapcheon, Seomjin river, Juam, and Yongdam) and 4 gauging stations (Osucheon, Mihocheon, Maruek, and Chogang) respectively. The calibration results showed that the coefficient of determination (R2) was 0.76 in average (0.66 to 0.84) and the Nash-Sutcliffe model efficiency was 0.62 in average (0.52 to 0.72). Based on the 2010s (2006~2015) weather condition for the whole period, the streamflow impact was estimated by applying GIS data for each decade (1980s: 1976~1985, 1990s: 1986~1995, 2000s: 1996~2005, 2010s: 2006~2015). The results showed that the 2010s averaged-wet streamflow (Q95) showed decrease of 4.1~6.3%, the 2010s averaged-normal streamflow (Q185) showed decreased of 6.7~9.1% and the 2010s averaged-drought streamflow (Q355) showed decrease of 8.4~10.4% compared to 1980s streamflows respectively on the whole. During 1975~2015, the increase of groundwater use covered 40.5% contribution and the next was forest growth with 29.0% contribution among the 5 influencing factors.

Channel Changes and Effect of Flow Pulses on Hydraulic Geometry Downstream of the Hapcheon Dam (합천댐 하류 하천지형 변화 예측 및 흐름파가 수리기하 변화에 미치는 영향)

  • Shin, Young-Ho;Julien, Pierre Y.
    • Journal of Korea Water Resources Association
    • /
    • v.42 no.7
    • /
    • pp.579-589
    • /
    • 2009
  • Hwang River in South Korea, has experienced channel adjustments due to dam construction. Hapcheon main dam and re-regulation dam. The reach below the re-regulation dam (45 km long) changed in flow regime, channel width, bed material distribution, vegetation expansion, and island formation after dam construction. The re-regulation dam dramatically reduced annual peak flow from 654.7 $m^3$/s to 126.3 $m^3$/s and trapped the annual 591 thousand $m^3$ of sediment load formerly delivered from the upper watershed since the completion of the dam in 1989. An analysis of a time series of aerial photographs taken in 1982, 1993, and 2004 showed that non-vegetated active channel width narrowed an average of 152 m (47% of 1982) and non-vegetated active channel area decreased an average of 6.6 km2 (44% of 1982) between 1982 and 2004, with most narrowing and decreasing occurring after dam construction. The effects of daily pulses of water from peak hydropower generation and sudden sluice gate operations are investigated downstream of Hapcheon Dam in South Korea. The study reach is 45 km long from the Hapcheon re-regulation Dam to the confluence with the Nakdong River. An analysis of a time series of aerial photographs taken in 1982, 1993, and 2004 showed that the non-vegetated active channel width narrowed an average of 152 m (47% reduction since 1982). The non-vegetated active channel area also decreased an average of 6.6 $km^2$ (44% reduction since 1982) between 1982 and 2004, with most changes occurring after dam construction. The average median bed material size increased from 1.07 mm in 1983 to 5.72 mm in 2003, and the bed slope of the reach decreased from 0.000943 in 1983 to 0.000847 in 2003. The riverbed vertical degradation is approximately 2.6 m for a distance of 20 km below the re-regulation dam. It is expected from the result of the unsteady sediment transport numerical model (GSTAR-1D) steady simulations that the thalweg elevation will reach a stable condition around 2020. The model also confirms the theoretical prediction that sediment transport rates from daily pulses and flood peaks are 21 % and 15 % higher than their respective averages.

Bias Correction for GCM Long-term Prediction using Nonstationary Quantile Mapping (비정상성 분위사상법을 이용한 GCM 장기예측 편차보정)

  • Moon, Soojin;Kim, Jungjoong;Kang, Boosik
    • Journal of Korea Water Resources Association
    • /
    • v.46 no.8
    • /
    • pp.833-842
    • /
    • 2013
  • The quantile mapping is utilized to reproduce reliable GCM(Global Climate Model) data by correct systematic biases included in the original data set. This scheme, in general, projects the Cumulative Distribution Function (CDF) of the underlying data set into the target CDF assuming that parameters of target distribution function is stationary. Therefore, the application of stationary quantile mapping for nonstationary long-term time series data of future precipitation scenario computed by GCM can show biased projection. In this research the Nonstationary Quantile Mapping (NSQM) scheme was suggested for bias correction of nonstationary long-term time series data. The proposed scheme uses the statistical parameters with nonstationary long-term trends. The Gamma distribution was assumed for the object and target probability distribution. As the climate change scenario, the 20C3M(baseline scenario) and SRES A2 scenario (projection scenario) of CGCM3.1/T63 model from CCCma (Canadian Centre for Climate modeling and analysis) were utilized. The precipitation data were collected from 10 rain gauge stations in the Han-river basin. In order to consider seasonal characteristics, the study was performed separately for the flood (June~October) and nonflood (November~May) seasons. The periods for baseline and projection scenario were set as 1973~2000 and 2011~2100, respectively. This study evaluated the performance of NSQM by experimenting various ways of setting parameters of target distribution. The projection scenarios were shown for 3 different periods of FF scenario (Foreseeable Future Scenario, 2011~2040 yr), MF scenario (Mid-term Future Scenario, 2041~2070 yr), LF scenario (Long-term Future Scenario, 2071~2100 yr). The trend test for the annual precipitation projection using NSQM shows 330.1 mm (25.2%), 564.5 mm (43.1%), and 634.3 mm (48.5%) increase for FF, MF, and LF scenarios, respectively. The application of stationary scheme shows overestimated projection for FF scenario and underestimated projection for LF scenario. This problem could be improved by applying nonstationary quantile mapping.

Application of deep learning method for decision making support of dam release operation (댐 방류 의사결정지원을 위한 딥러닝 기법의 적용성 평가)

  • Jung, Sungho;Le, Xuan Hien;Kim, Yeonsu;Choi, Hyungu;Lee, Giha
    • Journal of Korea Water Resources Association
    • /
    • v.54 no.spc1
    • /
    • pp.1095-1105
    • /
    • 2021
  • The advancement of dam operation is further required due to the upcoming rainy season, typhoons, or torrential rains. Besides, physical models based on specific rules may sometimes have limitations in controlling the release discharge of dam due to inherent uncertainty and complex factors. This study aims to forecast the water level of the nearest station to the dam multi-timestep-ahead and evaluate the availability when it makes a decision for a release discharge of dam based on LSTM (Long Short-Term Memory) of deep learning. The LSTM model was trained and tested on eight data sets with a 1-hour temporal resolution, including primary data used in the dam operation and downstream water level station data about 13 years (2009~2021). The trained model forecasted the water level time series divided by the six lead times: 1, 3, 6, 9, 12, 18-hours, and compared and analyzed with the observed data. As a result, the prediction results of the 1-hour ahead exhibited the best performance for all cases with an average accuracy of MAE of 0.01m, RMSE of 0.015 m, and NSE of 0.99, respectively. In addition, as the lead time increases, the predictive performance of the model tends to decrease slightly. The model may similarly estimate and reliably predicts the temporal pattern of the observed water level. Thus, it is judged that the LSTM model could produce predictive data by extracting the characteristics of complex hydrological non-linear data and can be used to determine the amount of release discharge from the dam when simulating the operation of the dam.

An Analytical Solution of Flow and Progressive Wave-Induced Residual Pore Water Pressure in Seabed (흐름과 진행파에 의한 해저지반 내 잔류간극수압의 해석해)

  • Lee, Kwang-Ho;Kim, Dong-Wook;Kang, Gi-Chun;Kim, Do-Sam;Kim, Tae-Hyung
    • Journal of the Korean Geotechnical Society
    • /
    • v.31 no.7
    • /
    • pp.13-28
    • /
    • 2015
  • This study extended the Lee et al.'s (2015a) solution which improved the existing analytical solution for prediction of the residual pore water pressure into progressive wave and flow coexisting field. At this time, the variation of incident wave period and wave length should be incorporated to Lee et al.'s (2015a) analytical solution, which does not consider flow. For the case of infinite thickness, the new analytical solution using Fourier series was compared to the analytical solution using Laplace transformation proposed by Jeng and Seymour (2007). It was verified that the new solution was identical to the Jeng and Seymour's solution. After verification of the new analytical solution, the residual pore water pressure head was examined closely under various given values of flow velocity's magnitude, direction, incident wave's period and seabed thickness. In each proposed analytical solution, asymptotic approach to shallow depth with the changes in the soil thickness within finite soil thickness was found possible, but not to infinite depth. It is also identified that there exists a discrepancy case between the results obtained from the finite and the infinite seabed thicknesses even on the same soil depth.

Analysis of Global Shipping Market Status and Forecasting the Container Freight Volume of Busan New port using Time-series Model (글로벌 해운시장 현황 분석 및 시계열 모형을 이용한 부산 신항 컨테이너 물동량 예측에 관한 연구)

  • JO, Jun-Ho;Byon, Je-Seop;Kim, Hee-Cheul
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.10 no.4
    • /
    • pp.295-303
    • /
    • 2017
  • In this paper, we analyze the trends of the international shipping market and the domestic and foreign factors of the crisis of the domestic shipping market, and identify the characteristics of the recovery of the Busan New Port trade volume which has decreased since the crisis of the domestic shipping market We quantitatively analyzed the future volume of Busan New Port and analyzed the trends of the prediction and recovery trends. As a result of analyzing Busan New Port container cargo volume by using big data analysis tool R, the variation of Busan New Cargo container cargo volume was estimated by ARIMA model (1,0,1) (1,0,1)[12] Estimation error, AICc and BIC were the most optimal ARIMA models. Therefore, we estimated the estimated value of Busan New Port trade for 36 months by using ARIMA (1, 0, 1)[12], which is the optimal model of Busan New Port trade, and estimated 13,157,184 TEU, 13,418,123 TEU, 13,539,884 TEU, and 4,526,406 TEU, respectively, indicating that it increased by about 2%, 2%, and 1%.