• Journal of the Korean Society of Systems Engineering
• /
• v.18 no.2
• /
• pp.75-93
• /
• 2022

Machine learning (ML) data-driven meta-model is proposed as a surrogate model to reduce the excessive computational cost of the physics-based model and facilitate the real-time prediction of a nuclear power plant's transient response. To forecast the transient response three machine learning (ML) meta-models based on recurrent neural networks (RNNs); specifically, Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), and a sequence combination of Convolutional Neural Network (CNN) and LSTM are developed. The chosen accident scenario is a control element assembly withdrawal at power concurrent with the Loss Of Offsite Power (LOOP). The transient response was obtained using the best estimate thermal hydraulics code, MARS-KS, and cross-validated against the Design and control document (DCD). DAKOTA software is loosely coupled with MARS-KS code via a python interface to perform the Best Estimate Plus Uncertainty Quantification (BEPU) analysis and generate a time series database of the system response to train, test and validate the ML meta-models. Key uncertain parameters identified as required by the CASU methodology were propagated using the non-parametric Monte-Carlo (MC) random propagation and Latin Hypercube Sampling technique until a statistically significant database (181 samples) as required by Wilk's fifth order is achieved with 95% probability and 95% confidence level. The three ML RNN models were built and optimized with the help of the Talos tool and demonstrated excellent performance in forecasting the most probable NPP transient response. This research was guided by the Systems Engineering (SE) approach for the systematic and efficient planning and execution of the research.

• ETRI Journal
• /
• v.41 no.3
• /
• pp.371-382
• /
• 2019

Machine reading comprehension is the task of understanding a given context and finding the correct response in that context. A simple recurrent unit (SRU) is a model that solves the vanishing gradient problem in a recurrent neural network (RNN) using a neural gate, such as a gated recurrent unit (GRU) and long short-term memory (LSTM); moreover, it removes the previous hidden state from the input gate to improve the speed compared to GRU and LSTM. A self-matching network, used in R-Net, can have a similar effect to coreference resolution because the self-matching network can obtain context information of a similar meaning by calculating the attention weight for its own RNN sequence. In this paper, we construct a dataset for Korean machine reading comprehension and propose an $S^2-Net$ model that adds a self-matching layer to an encoder RNN using multilayer SRU. The experimental results show that the proposed $S^2-Net$ model has performance of single 68.82% EM and 81.25% F1, and ensemble 70.81% EM, 82.48% F1 in the Korean machine reading comprehension test dataset, and has single 71.30% EM and 80.37% F1 and ensemble 73.29% EM and 81.54% F1 performance in the SQuAD dev dataset.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.19 no.4
• /
• pp.161-167
• /
• 2019

Recently, demand forecasting techniques have been actively studied due to interest in stable power supply with surging power demand, and increase in spread of smart meters that enable real-time power measurement. In this study, we proceeded the deep learning prediction model experiments which learns actual measured power usage data of home and outputs the forecasting result. And we proceeded pre-processing with moving average method. The predicted value made by the model is evaluated with the actual measured data. Through this forecasting, it is possible to lower the power supply reserve ratio and reduce the waste of the unused power. In this paper, we conducted experiments on three types of networks: Multi Layer Perceptron (MLP), Recurrent Neural Network (RNN), and Long Short Term Memory (LSTM) and we evaluate the results of each scheme. Evaluation is conducted with following method: MSE(Mean Squared Error) method and MAE(Mean Absolute Error).

• Journal of Advanced Navigation Technology
• /
• v.25 no.5
• /
• pp.409-414
• /
• 2021

The growing concerns on the emission of particulate matter has prompted a demand for highly reliable particulate matter forecasting. Currently, several studies on particulate matter prediction use various deep learning algorithms. In this study, we compared the predictive performances of typical neural networks used for particulate matter prediction. We used deep neural network(DNN), recurrent neural network, and long short-term memory algorithms to design an optimal predictive model on the basis of a hyperparameter search. The results of a comparative analysis of the predictive performances of the models indicate that the variation trend of the actual and predicted values generally showed a good performance. In the analysis based on the root mean square error and accuracy, the DNN-based prediction model showed a higher reliability for prediction errors compared with the other prediction models.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.13 no.4
• /
• pp.2060-2077
• /
• 2019

Recently, mobile healthcare services have attracted significant attention because of the emerging development and supply of diverse wearable devices. Smartwatches and health bands are the most common type of mobile-based wearable devices and their market size is increasing considerably. However, simple value comparisons based on accumulated data have revealed certain problems, such as the standardized nature of health management and the lack of personalized health management service models. The convergence of information technology (IT) and biotechnology (BT) has shifted the medical paradigm from continuous health management and disease prevention to the development of a system that can be used to provide ground-based medical services regardless of the user's location. Moreover, the IT-BT convergence has necessitated the development of lifestyle improvement models and services that utilize big data analysis and machine learning to provide mobile healthcare-based personal health management and disease prevention information. Users' health data, which are specific as they change over time, are collected by different means according to the users' lifestyle and surrounding circumstances. In this paper, we propose a prediction model of user physical activity that uses data characteristics-based long short-term memory (DC-LSTM) recurrent neural networks (RNNs). To provide personalized services, the characteristics and surrounding circumstances of data collectable from mobile host devices were considered in the selection of variables for the model. The data characteristics considered were ease of collection, which represents whether or not variables are collectable, and frequency of occurrence, which represents whether or not changes made to input values constitute significant variables in terms of activity. The variables selected for providing personalized services were activity, weather, temperature, mean daily temperature, humidity, UV, fine dust, asthma and lung disease probability index, skin disease probability index, cadence, travel distance, mean heart rate, and sleep hours. The selected variables were classified according to the data characteristics. To predict activity, an LSTM RNN was built that uses the classified variables as input data and learns the dynamic characteristics of time series data. LSTM RNNs resolve the vanishing gradient problem that occurs in existing RNNs. They are classified into three different types according to data characteristics and constructed through connections among the LSTMs. The constructed neural network learns training data and predicts user activity. To evaluate the proposed model, the root mean square error (RMSE) was used in the performance evaluation of the user physical activity prediction method for which an autoregressive integrated moving average (ARIMA) model, a convolutional neural network (CNN), and an RNN were used. The results show that the proposed DC-LSTM RNN method yields an excellent mean RMSE value of 0.616. The proposed method is used for predicting significant activity considering the surrounding circumstances and user status utilizing the existing standardized activity prediction services. It can also be used to predict user physical activity and provide personalized healthcare based on the data collectable from mobile host devices.

• KIISE Transactions on Computing Practices
• /
• v.23 no.5
• /
• pp.322-327
• /
• 2017

The classification problem in the field of Natural Language Processing has been studied for a long time. Continuing forward with our previous research, which classifies large-scale text using Convolutional Neural Networks (CNN), we implemented Recurrent Neural Networks (RNN), Long-Short Term Memory (LSTM) and Gated Recurrent Units (GRU). The experiment's result revealed that the performance of classification algorithms was Multinomial Naïve Bayesian Classifier < Support Vector Machine (SVM) < LSTM < CNN < GRU, in order. The result can be interpreted as follows: First, the result of CNN was better than LSTM. Therefore, the text classification problem might be related more to feature extraction problem than to natural language understanding problems. Second, judging from the results the GRU showed better performance in feature extraction than LSTM. Finally, the result that the GRU was better than CNN implies that text classification algorithms should consider feature extraction and sequential information. We presented the results of fine-tuning in deep neural networks to provide some intuition regard natural language processing to future researchers.

• The Journal of Korea Institute of Information, Electronics, and Communication Technology
• /
• v.11 no.2
• /
• pp.204-208
• /
• 2018

In this paper, we proposed and evaluated the time series deep learning prediction model for learning fluctuation pattern of stock price. Recurrent neural networks, which can store previous information in the hidden layer, are suitable for the stock price prediction model, which is time series data. In order to maintain the long - term dependency by solving the gradient vanish problem in the recurrent neural network, we use LSTM with small memory inside the recurrent neural network. Furthermore, we proposed the stock price prediction model using bidirectional LSTM recurrent neural network in which the hidden layer is added in the reverse direction of the data flow for solving the limitation of the tendency of learning only based on the immediately preceding pattern of the recurrent neural network. In this experiment, we used the Tensorflow to learn the proposed stock price prediction model with stock price and trading volume input. In order to evaluate the performance of the stock price prediction, the mean square root error between the real stock price and the predicted stock price was obtained. As a result, the stock price prediction model using bidirectional LSTM recurrent neural network has improved prediction accuracy compared with unidirectional LSTM recurrent neural network.

• Geophysics and Geophysical Exploration
• /
• v.26 no.3
• /
• pp.126-137
• /
• 2023

In this study, a recurrent neural network (RNN) was employed as a methodological approach to classify dolphin click signals derived from ocean monitoring data. To improve the accuracy of click signal classification, the single time series data were transformed into fractional domains using fractional Fourier transform to expand its features. Transformed data were used as input for three RNN models: long short-term memory (LSTM), gated recurrent unit (GRU), and bidirectional LSTM (BiLSTM), which were compared to determine the optimal network for the classification of signals. Because the fractional Fourier transform displayed different characteristics depending on the chosen angle parameter, the optimal angle range for each RNN was first determined. To evaluate network performance, metrics such as accuracy, precision, recall, and F1-score were employed. Numerical experiments demonstrated that all three networks performed well, however, the BiLSTM network outperformed LSTM and GRU in terms of learning results. Furthermore, the BiLSTM network provided lower misclassification than the other networks and was deemed the most practically appliable to field data.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.19 no.1
• /
• pp.29-35
• /
• 2019

As the fourth industrial revolution is emerging, many companies are increasingly interested in smart factories and the importance of sensors is being emphasized. In the case that sensors for collecting sensing data fail, the plant could not be optimized and further it could not be operated properly, which may incur a financial loss. For this purpose, it is necessary to diagnose the status of sensors to prevent sensor' fault. In the paper, we propose a scheme to diagnose digital-sensor' fault by analyzing the rising time and falling time of digital sensors through the LSTM(Long Short Term Memory) of Deep Learning RNN algorithm. Experimental results of the proposed scheme are compared with those of rule-based fault diagnosis algorithm in terms of AUC(Area Under the Curve) of accuracy and ROC(Receiver Operating Characteristic) curve. Experimental results show that the proposed system has better and more stable performance than the rule-based fault diagnosis algorithm.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.41 no.5
• /
• pp.592-601
• /
• 2016

The recurrent neural network (RNN) is a deep learning model which is suitable to sequential or length-variable data. The Long Short-Term Memory (LSTM) mitigates the vanishing gradient problem of RNNs so that LSTM can maintain the long-term dependency among the constituents of the given input sequence. In this paper, we propose a LSTM based language model which can predict following words of a given incomplete sentence to generate a complete sentence. To evaluate our method, we trained our model using multiple Korean corpora then generated the incomplete part of Korean sentences. The result shows that our language model was able to generate the fluent Korean sentences. We also show that the word based model generated better sentences compared to the other settings.