• Title/Summary/Keyword: artificial neural network (ANN)

Search Result 1,048, Processing Time 0.048 seconds

COMPARISON OF LINEAR AND NON-LINEAR NIR CALIBRATION METHODS USING LARGE FORAGE DATABASES

  • Berzaghi, Paolo;Flinn, Peter C.;Dardenne, Pierre;Lagerholm, Martin;Shenk, John S.;Westerhaus, Mark O.;Cowe, Ian A.
    • Proceedings of the Korean Society of Near Infrared Spectroscopy Conference
    • /
    • 2001.06a
    • /
    • pp.1141-1141
    • /
    • 2001
  • The aim of the study was to evaluate the performance of 3 calibration methods, modified partial least squares (MPLS), local PLS (LOCAL) and artificial neural network (ANN) on the prediction of chemical composition of forages, using a large NIR database. The study used forage samples (n=25,977) from Australia, Europe (Belgium, Germany, Italy and Sweden) and North America (Canada and U.S.A) with information relative to moisture, crude protein and neutral detergent fibre content. The spectra of the samples were collected with 10 different Foss NIR Systems instruments, which were either standardized or not standardized to one master instrument. The spectra were trimmed to a wavelength range between 1100 and 2498 nm. Two data sets, one standardized (IVAL) and the other not standardized (SVAL) were used as independent validation sets, but 10% of both sets were omitted and kept for later expansion of the calibration database. The remaining samples were combined into one database (n=21,696), which was split into 75% calibration (CALBASE) and 25% validation (VALBASE). The chemical components in the 3 validation data sets were predicted with each model derived from CALBASE using the calibration database before and after it was expanded with 10% of the samples from IVAL and SVAL data sets. Calibration performance was evaluated using standard error of prediction corrected for bias (SEP(C)), bias, slope and R2. None of the models appeared to be consistently better across all validation sets. VALBASE was predicted well by all models, with smaller SEP(C) and bias values than for IVAL and SVAL. This was not surprising as VALBASE was selected from the calibration database and it had a sample population similar to CALBASE, whereas IVAL and SVAL were completely independent validation sets. In most cases, Local and ANN models, but not modified PLS, showed considerable improvement in the prediction of IVAL and SVAL after the calibration database had been expanded with the 10% samples of IVAL and SVAL reserved for calibration expansion. The effects of sample processing, instrument standardization and differences in reference procedure were partially confounded in the validation sets, so it was not possible to determine which factors were most important. Further work on the development of large databases must address the problems of standardization of instruments, harmonization and standardization of laboratory procedures and even more importantly, the definition of the database population.

  • PDF

Vulnerability Assessment of the Climate Change on the Water Environment of Juam Reservoir (기후변화에 따른 주암호 수환경 취약성 평가)

  • Yoon, Sung Wan;Chung, Se Woong;Park, Hyung Seok
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2015.05a
    • /
    • pp.519-519
    • /
    • 2015
  • 2007년 발간된 IPCC의 4차 평가보고서에서 자연재해, 환경, 해양, 농업, 생태계, 보건 등 다양한 부분에 미치는 기후변화의 영향에 대한 과학적 근거들이 제시되면서 기후변화는 현세기 범지구적인 화두로 대두되고 있다. 또한, 기후변화에 의한 지구 온난화는 대규모의 수문순환 과정에서의 변화들과 연관되어 담수자원은 기후변화에 대단히 취약하며 미래로 갈수록 악영향을 받을 것으로 6차 기술보고서에서 제시하고 있다. 특히 우리나라는 지구온난화가 전 지구적인 평균보다 급속하게 진행될 가능성이 높기 때문에 기후변화에 대한 담수자원 취약성이 더욱 클 것으로 예상된다. 따라서 지표수에 용수의존도가 높은 우리나라의 댐 저수지를 대상으로 기후변화에 따른 수환경 변화의 정확한 분석과 취약성 평가는 필수적이다. 본 연구에서는 SRES A1B 시나리오를 적용하여 기후변화가 주암호 저수지의 수환경 변화에 미치는 영향을 분석하였다. 지역스케일의 미래 기후시나리오 생산을 위해 인공신경망(Artificial Neural Network.,ANN)기법을 적용하여 예측인자(강우, 상대습도, 최고온도, 최저온도)에 대해 강우-유출모형에 적용이 가능한 지역스케일로 통계적 상세화를 수행하였으며, 이를 유역모델에 적용하여 저수지 유입부의 유출량 및 부하량을 예측하였다. 유역 모델의 결과를 토대로 저수지 운영모델에 저수지 유입부의 유출량을 적용하여 미래 기간의 방류량을 산정하였으며, 최종적으로 저수지 모델에 유입량, 유입부하량 및 방류량을 적용하여 저수지 내 오염 및 영양물질 순환 및 분포 예측을 통해서 기후변화가 저수지 수환경에 미치는 영향을 평가하였다. 기후변화 시나리오에 따른 상세기 후전망을 위해서 기후인자의 미래분석 기간은 (I)단계 구간(2011~2040년), (II)단계 구간(2041~2070년), (III) 단계 구간(2071~2100년)의 3개 구간으로 설정하여 수행하였으며, Baseline인 1991~2010년까지의 실측값과 모의 값을 비교하여 검증하였다. 강우량의 경우 Baseline 대비 미래로 갈수록 증가하는 것으로 전망되었으며, 2011년 대비 2100년에서 연강수량 6.4% 증가한 반면, 일최대강수량이 7.0% 증가하는 것으로 나타나 미래로 갈수록 집중호우의 발생가능성이 커질 것으로 예측되었다. 유역의 수문 수질변화 전망도 강수량 증가의 영향으로 주암댐으로 유입하는 총 유량이 Baseline 대비 증가 하였으며, 유사량 및 오염부하량도 증가하는 것으로 나타났다. 저수지 수환경 변화 예측결과 유입량이 증가함에 따라서 연평균 체류시간이 감소하였으며, 기온 및 유입수온 상승의 영향으로 (I)단계 구간대비 미래로 갈수록 상층 및 심층의 수온이 상승하는 것으로 나타났다. 연중 수온성층기간 역시 증가하는 것으로 나타났으며, 남조류는 (I)단계 구간 대비 (III)단계 구간으로 갈수록 출현시기가 빨라지며 농도 역시 증가하였다. 또한 풍수년, 평수년에 비해 갈수년에 남조류의 연평균농도 상승폭과 최고농도가 크게 나타나 미래로 갈수록 댐 유입량이 적은 해에 남조류로 인한 피해 발생 가능성이 높아질 것으로 예상된다.

  • PDF

Prediction of cyanobacteria harmful algal blooms in reservoir using machine learning and deep learning (머신러닝과 딥러닝을 이용한 저수지 유해 남조류 발생 예측)

  • Kim, Sang-Hoon;Park, Jun Hyung;Kim, Byunghyun
    • Journal of Korea Water Resources Association
    • /
    • v.54 no.spc1
    • /
    • pp.1167-1181
    • /
    • 2021
  • In relation to the algae bloom, four types of blue-green algae that emit toxic substances are designated and managed as harmful Cyanobacteria, and prediction information using a physical model is being also published. However, as algae are living organisms, it is difficult to predict according to physical dynamics, and not easy to consider the effects of numerous factors such as weather, hydraulic, hydrology, and water quality. Therefore, a lot of researches on algal bloom prediction using machine learning have been recently conducted. In this study, the characteristic importance of water quality factors affecting the occurrence of Cyanobacteria harmful algal blooms (CyanoHABs) were analyzed using the random forest (RF) model for Bohyeonsan Dam and Yeongcheon Dam located in Yeongcheon-si, Gyeongsangbuk-do and also predicted the occurrence of harmful blue-green algae using the machine learning and deep learning models and evaluated their accuracy. The water temperature and total nitrogen (T-N) were found to be high in common, and the occurrence prediction of CyanoHABs using artificial neural network (ANN) also predicted the actual values closely, confirming that it can be used for the reservoirs that require the prediction of harmful cyanobacteria for algal management in the future.

Improvement of multi layer perceptron performance using combination of gradient descent and harmony search for prediction of ground water level (지하수위 예측을 위한 경사하강법과 화음탐색법의 결합을 이용한 다층퍼셉트론 성능향상)

  • Lee, Won Jin;Lee, Eui Hoon
    • Journal of Korea Water Resources Association
    • /
    • v.55 no.11
    • /
    • pp.903-911
    • /
    • 2022
  • Groundwater, one of the resources for supplying water, fluctuates in water level due to various natural factors. Recently, research has been conducted to predict fluctuations in groundwater levels using Artificial Neural Network (ANN). Previously, among operators in ANN, Gradient Descent (GD)-based Optimizers were used as Optimizer that affect learning. GD-based Optimizers have disadvantages of initial correlation dependence and absence of solution comparison and storage structure. This study developed Gradient Descent combined with Harmony Search (GDHS), a new Optimizer that combined GD and Harmony Search (HS) to improve the shortcomings of GD-based Optimizers. To evaluate the performance of GDHS, groundwater level at Icheon Yullhyeon observation station were learned and predicted using Multi Layer Perceptron (MLP). Mean Squared Error (MSE) and Mean Absolute Error (MAE) were used to compare the performance of MLP using GD and GDHS. Comparing the learning results, GDHS had lower maximum, minimum, average and Standard Deviation (SD) of MSE than GD. Comparing the prediction results, GDHS was evaluated to have a lower error in all of the evaluation index than GD.

A Time Series Graph based Convolutional Neural Network Model for Effective Input Variable Pattern Learning : Application to the Prediction of Stock Market (효과적인 입력변수 패턴 학습을 위한 시계열 그래프 기반 합성곱 신경망 모형: 주식시장 예측에의 응용)

  • Lee, Mo-Se;Ahn, Hyunchul
    • Journal of Intelligence and Information Systems
    • /
    • v.24 no.1
    • /
    • pp.167-181
    • /
    • 2018
  • Over the past decade, deep learning has been in spotlight among various machine learning algorithms. In particular, CNN(Convolutional Neural Network), which is known as the effective solution for recognizing and classifying images or voices, has been popularly applied to classification and prediction problems. In this study, we investigate the way to apply CNN in business problem solving. Specifically, this study propose to apply CNN to stock market prediction, one of the most challenging tasks in the machine learning research. As mentioned, CNN has strength in interpreting images. Thus, the model proposed in this study adopts CNN as the binary classifier that predicts stock market direction (upward or downward) by using time series graphs as its inputs. That is, our proposal is to build a machine learning algorithm that mimics an experts called 'technical analysts' who examine the graph of past price movement, and predict future financial price movements. Our proposed model named 'CNN-FG(Convolutional Neural Network using Fluctuation Graph)' consists of five steps. In the first step, it divides the dataset into the intervals of 5 days. And then, it creates time series graphs for the divided dataset in step 2. The size of the image in which the graph is drawn is $40(pixels){\times}40(pixels)$, and the graph of each independent variable was drawn using different colors. In step 3, the model converts the images into the matrices. Each image is converted into the combination of three matrices in order to express the value of the color using R(red), G(green), and B(blue) scale. In the next step, it splits the dataset of the graph images into training and validation datasets. We used 80% of the total dataset as the training dataset, and the remaining 20% as the validation dataset. And then, CNN classifiers are trained using the images of training dataset in the final step. Regarding the parameters of CNN-FG, we adopted two convolution filters ($5{\times}5{\times}6$ and $5{\times}5{\times}9$) in the convolution layer. In the pooling layer, $2{\times}2$ max pooling filter was used. The numbers of the nodes in two hidden layers were set to, respectively, 900 and 32, and the number of the nodes in the output layer was set to 2(one is for the prediction of upward trend, and the other one is for downward trend). Activation functions for the convolution layer and the hidden layer were set to ReLU(Rectified Linear Unit), and one for the output layer set to Softmax function. To validate our model - CNN-FG, we applied it to the prediction of KOSPI200 for 2,026 days in eight years (from 2009 to 2016). To match the proportions of the two groups in the independent variable (i.e. tomorrow's stock market movement), we selected 1,950 samples by applying random sampling. Finally, we built the training dataset using 80% of the total dataset (1,560 samples), and the validation dataset using 20% (390 samples). The dependent variables of the experimental dataset included twelve technical indicators popularly been used in the previous studies. They include Stochastic %K, Stochastic %D, Momentum, ROC(rate of change), LW %R(Larry William's %R), A/D oscillator(accumulation/distribution oscillator), OSCP(price oscillator), CCI(commodity channel index), and so on. To confirm the superiority of CNN-FG, we compared its prediction accuracy with the ones of other classification models. Experimental results showed that CNN-FG outperforms LOGIT(logistic regression), ANN(artificial neural network), and SVM(support vector machine) with the statistical significance. These empirical results imply that converting time series business data into graphs and building CNN-based classification models using these graphs can be effective from the perspective of prediction accuracy. Thus, this paper sheds a light on how to apply deep learning techniques to the domain of business problem solving.

Development of a Freeway Travel Time Forecasting Model for Long Distance Section with Due Regard to Time-lag (시간처짐현상을 고려한 장거리구간 통행시간 예측 모형 개발)

  • 이의은;김정현
    • Journal of Korean Society of Transportation
    • /
    • v.20 no.4
    • /
    • pp.51-61
    • /
    • 2002
  • In this dissertation, We demonstrated the Travel Time forecasting model in the freeway of multi-section with regard of drives' attitude. Recently, the forecasted travel time that is furnished based on expected travel time data and advanced experiment isn't being able to reflect the time-lag phenomenon specially in case of long distance trip, so drivers don't believe any more forecasted travel time. And that's why the effects of ATIS(Advanced Traveler Information System) are reduced. Therefore, in this dissertation to forecast the travel time of the freeway of multi-section reflecting the time-lag phenomenon & the delay of tollgate, we used traffic volume data & TCS data that are collected by Korea Highway Cooperation. Also keep the data of mixed unusual to applicate real system. The applied model for forecasting is consisted of feed-forward structure which has three input units & two output units and the back-propagation is utilized as studying method. Furthermore, the optimal alternative was chosen through the twelve alternative ideas which is composed of the unit number of hidden-layer & repeating number which affect studying speed & forecasting capability. In order to compare the forecasting capability of developed ANN model. the algorithm which are currently used as an information source for freeway travel time. During the comparison with reference model, MSE, MARE, MAE & T-test were executed, as the result, the model which utilized the artificial neural network performed more superior forecasting capability among the comparison index. Moreover, the calculated through the particularity of data structure which was used in this experiment.

Effect of Change in Hydrological Environment by Climate Change on River Water Quality in Nam River Watershed (기후변화에 따른 남강유역의 수문환경의 변화가 하천수질에 미치는 영향)

  • Kang, Ji Yoon;Kim, Young Do;Kang, Boo Sik
    • Journal of Korea Water Resources Association
    • /
    • v.46 no.8
    • /
    • pp.873-884
    • /
    • 2013
  • In Korea, the rainfall is concentrated in summer under the influence of monsoon climate. Thus, even a small climate change can be significant problems in water resources. As a result, a lot of attention has been focused on climate changes and a number of researches have been conducted in a manner commensurate with the attention to the climate change. This study is intended to forecast the changes in the flow and water quality of the Nam river resulting from the future climate changes in the Nam river basin using a watershed and water quality model. An SWAT model, as a watershed hydrologic model, was established after estimating a climate scenario using an artificial neural network method, and the established model was verified and adjusted using date from the Ministry of Environment to evaluate the applicability of the model. As a consequence, $R^2$ showed more than 0.7 in the simulation test, which satisfies the minimum required level. Results from the SWAT model and the future Namgang dam discharge calculated by HEC-ResSIM is used as input date for QUALKO. The results showed a huge variation in BOD depending on the annual flow of the river, which recorded a maximum difference of 2 mg/L between a rainy season and a dry season. It can be deduced that because rainfall and the runoff of a basin significantly account for the water quality of a river, higher water concentrations are recorded in a dry season in which the flow is not as much as that in a rainy season. It also can be said that water should be reserved in advance to secure water in the Nam river downstream for a dry season and be controlled in an effective and efficient manner to provide better water quality.

Comparative analysis of water surface spectral characteristics based on hyperspectral images for chlorophyll-a estimation in Namyang estuarine reservoir and Baekje weir (남양호와 백제보의 Chlorophyll-a 산정을 위한 초분광 영상기반 수체분광특성 비교 분석)

  • Jang, Wonjin;Kim, Jinuk;Kim, Jinhwi;Nam, Guisook;Kang, Euetae;Park, Yongeun;Kim, Seongjoon
    • Journal of Korea Water Resources Association
    • /
    • v.56 no.2
    • /
    • pp.91-101
    • /
    • 2023
  • In this study, we estimated the concentration of chlorophyll-a (Chl-a) using hyperspectral water surface reflectance in an inland weir (Baekjae weir) and estuarine reservoir (Namyang Reservoir) for monitoring the occurrence of algae in freshwater in South Korea. The hyperspectral reflectance was measured by aircraft in Baekjae Weir (BJW) from 2016 to 2017, and a drone in Namyang Reservoir (NYR) from 2020 to 2021. The 30 reflectance bands (BJW: 400-530, 620-680, 710-730, 760-790 nm, NYR: 400-430, 655-680, 740-800 nm) that were highly related to Chl-a concentration were selected using permutation importance. Artificial neural network based Chl-a estimation model was developed using the selected reflectance in both water bodies. And the performance of the model was evaluated with the coefficient of determination (R2), the root mean square error (RMSE), and the mean absolute error (MAE). The performance evaluation results of the Chl-a estimation model for each watershed was R2: 0.63, 0.82, RMSE: 9.67, 6.99, and MAE: 11.25, 8.48, respectively. The developed Chl-a model of this study may be used as foundation tool for the optimal management of freshwater algal blooms in the future.