• Journal of Information Processing Systems
• /
• v.17 no.4
• /
• pp.818-833
• /
• 2021

To address the problems of low precision rate, insufficient feature extraction, and poor contextual ability in existing text sentiment analysis methods, a mixed model account of a CNN-BiLSTM-TE (convolutional neural network, bidirectional long short-term memory, and topic extraction) model was proposed. First, Chinese text data was converted into vectors through the method of transfer learning by Word2Vec. Second, local features were extracted by the CNN model. Then, contextual information was extracted by the BiLSTM neural network and the emotional tendency was obtained using softmax. Finally, topics were extracted by the term frequency-inverse document frequency and K-means. Compared with the CNN, BiLSTM, and gate recurrent unit (GRU) models, the CNN-BiLSTM-TE model's F1-score was higher than other models by 0.0147, 0.006, and 0.0052, respectively. Then compared with CNN-LSTM, LSTM-CNN, and BiLSTM-CNN models, the F1-score was higher by 0.0071, 0.0038, and 0.0049, respectively. Experimental results showed that the CNN-BiLSTM-TE model can effectively improve various indicators in application. Lastly, performed scalability verification through a takeaway dataset, which has great value in practical applications.

• Journal of the Korean Society of Marine Environment & Safety
• /
• v.29 no.6
• /
• pp.543-551
• /
• 2023

As climate change continues to prompt an increasing demand for advancements in disaster and safety management technologies to address abnormal high water temperatures, typhoons, floods, and droughts, sea surface temperature has emerged as a pivotal factor for swiftly assessing the impacts of summer harmful algal blooms in the seas surrounding Korean Peninsula and the formation and dissipation of cold water along the East Coast of Korea. Therefore, this study sought to gauge predictive performance by leveraging statistical methods and deep learning algorithms to harness sea surface temperature data effectively for marine anomaly research. The sea surface temperature data employed in the predictions spans from 2018 to 2022 and originates from the Heuksando Tidal Observatory. Both traditional statistical ARIMA methods and advanced deep learning models, including long short-term memory (LSTM) and gated recurrent unit (GRU), were employed. Furthermore, prediction performance was evaluated using the attention LSTM technique. The technique integrated an attention mechanism into the sequence-to-sequence (s2s), further augmenting the performance of LSTM. The results showed that the attention LSTM model outperformed the other models, signifying its superior predictive performance. Additionally, fine-tuning hyperparameters can improve sea surface temperature performance.

• Journal of the Korea Society of Computer and Information
• /
• v.26 no.10
• /
• pp.179-184
• /
• 2021

In this paper, we propose the deep learning-based neural network model to predict bunker price. In the shipping industry, since fuel oil accounts for the largest portion of ship operation costs and its price is highly volatile, so companies can secure market competitiveness by making fuel oil purchasing decisions based on rational and scientific method. In this paper, short-term predictive analysis of HSFO 380CST in Singapore is conducted by using three recurrent neural network models like RNN, LSTM, and GRU. As a result, first, the forecasting performance of RNN models is better than LSTM and GRUs using long-term memory, and thus the predictive contribution of long-term information is low. Second, since the predictive performance of recurrent neural network models is superior to the previous studies using econometric models, it is confirmed that the recurrent neural network models should consider nonlinear properties of bunker price. The result of this paper will be helpful to improve the decision quality of bunker purchasing.

• Journal of the Korea Society of Computer and Information
• /
• v.28 no.8
• /
• pp.21-30
• /
• 2023

The study proposes a model that utilizes Python-based deep learning text classification techniques to detect the legality of illegal financial advertising posts on the internet. These posts aim to promote unlawful financial activities, including the trading of bank accounts, credit card fraud, cashing out through mobile payments, and the sale of personal credit information. Despite the efforts of financial regulatory authorities, the prevalence of illegal financial activities persists. By applying this proposed model, the intention is to aid in identifying and detecting illicit content in internet-based illegal financial advertisining, thus contributing to the ongoing efforts to combat such activities. The study utilizes convolutional neural networks(CNN) and recurrent neural networks(RNN, LSTM, GRU), which are commonly used text classification techniques. The raw data for the model is based on manually confirmed regulatory judgments. By adjusting the hyperparameters of the Korean natural language processing and deep learning models, the study has achieved an optimized model with the best performance. This research holds significant meaning as it presents a deep learning model for discerning internet illegal financial advertising, which has not been previously explored. Additionally, with an accuracy range of 91.3% to 93.4% in a deep learning model, there is a hopeful anticipation for the practical application of this model in the task of detecting illicit financial advertisements, ultimately contributing to the eradication of such unlawful financial advertisements.

• Communications for Statistical Applications and Methods
• /
• v.28 no.4
• /
• pp.329-338
• /
• 2021

Deep learning methods have been developed, used in various fields, and they have shown outstanding performances in many cases. Many studies predicted a daily stock return, a classic example of time-series data, using deep learning methods. We also tried to apply deep learning methods to Korea's stock market data. We used Korea's stock market index (KOSPI) and several individual stocks to forecast daily returns and directions. We compared several deep learning models with other machine learning methods, including random forest and XGBoost. In regression, long short term memory (LSTM) and gated recurrent unit (GRU) models are better than other prediction models. For the classification applications, there is no clear winner. However, even the best deep learning models cannot predict significantly better than the simple base model. We believe that it is challenging to predict daily stock return data even if we use the latest deep learning methods.

• Journal of the Korea Society of Computer and Information
• /
• v.27 no.8
• /
• pp.49-59
• /
• 2022

In this paper, we designed a new enzyme function prediction model PSCREM based on a study that compared and evaluated CNN and LSTM/GRU models, which are the most widely used deep learning models in the field of predicting functions and structures using protein sequences in 2020, under the same conditions. Sequence evolution information was used to preserve detailed patterns which would miss in CNN convolution, and the relationship information between amino acids with functional significance was extracted through overlapping RNNs. It was referenced to feature map production. The RNN family of algorithms used in small CNN-RNN models are LSTM algorithms and GRU algorithms, which are usually stacked two to three times over 100 units, but in this paper, small RNNs consisting of 10 and 20 units are overlapped. The model used the PSSM profile, which is transformed from protein sequence data. The experiment proved 86.4% the performance for the problem of predicting the main classes of enzyme number, and it was confirmed that the performance was 84.4% accurate up to the sub-sub classes of enzyme number. Thus, PSCREM better identifies unique patterns related to protein function through overlapped RNN, and Overlapped RNN is proposed as a novel methodology for protein function and structure prediction extraction.

• Journal of the Korean Society of Systems Engineering
• /
• v.18 no.2
• /
• pp.75-93
• /
• 2022

Machine learning (ML) data-driven meta-model is proposed as a surrogate model to reduce the excessive computational cost of the physics-based model and facilitate the real-time prediction of a nuclear power plant's transient response. To forecast the transient response three machine learning (ML) meta-models based on recurrent neural networks (RNNs); specifically, Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), and a sequence combination of Convolutional Neural Network (CNN) and LSTM are developed. The chosen accident scenario is a control element assembly withdrawal at power concurrent with the Loss Of Offsite Power (LOOP). The transient response was obtained using the best estimate thermal hydraulics code, MARS-KS, and cross-validated against the Design and control document (DCD). DAKOTA software is loosely coupled with MARS-KS code via a python interface to perform the Best Estimate Plus Uncertainty Quantification (BEPU) analysis and generate a time series database of the system response to train, test and validate the ML meta-models. Key uncertain parameters identified as required by the CASU methodology were propagated using the non-parametric Monte-Carlo (MC) random propagation and Latin Hypercube Sampling technique until a statistically significant database (181 samples) as required by Wilk's fifth order is achieved with 95% probability and 95% confidence level. The three ML RNN models were built and optimized with the help of the Talos tool and demonstrated excellent performance in forecasting the most probable NPP transient response. This research was guided by the Systems Engineering (SE) approach for the systematic and efficient planning and execution of the research.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2021.10a
• /
• pp.297-299
• /
• 2021

Predicting future productivity of tight oil is an important task for analyzing residual oil recovery and reservoir behavior. In general, productivity prediction is made using the decline curve analysis(DCA). In this study, we intend to propose an effective model for predicting future production using deep learning-based recurrent neural networks(RNN), LSTM, and GRU algorithms. As input variables, the main parameters are oil, gas, water, which are calculated during the production of tight oil, and the type curve calculated through various cluster analyzes. the output variable is the monthly oil production. Existing empirical models, the DCA and RNN models, were compared, and an optimal model was derived through hyperparameter tuning to improve the predictive performance of the model.

• International conference on construction engineering and project management
• /
• 2022.06a
• /
• pp.814-822
• /
• 2022

The deployment of sensors for Structural Health Monitoring requires a complicated network arrangement, ground truthing, and calibration for validating sensor performance periodically. Any conventional sensor on a structural element is also subjected to static and dynamic vertical loadings in conjunction with other environmental factors, such as brightness, noise, temperature, and humidity. A structural model with strain gauges was built and tested to get realistic sensory information. This paper investigates different deep learning architectures and algorithms, including unsupervised, autoencoder, and supervised methods, to benchmark blind drift calibration methods using deep learning. It involves a fully connected neural network (FCNN), a long short-term memory (LSTM), and a gated recurrent unit (GRU) to address the blind drift calibration problem (i.e., performing calibrations of installed sensors when ground truth is not available). The results show that the supervised methods perform much better than unsupervised methods, such as an autoencoder, when ground truths are available. Furthermore, taking advantage of time-series information, the GRU model generates the most precise predictions to remove the drift overall.

• ETRI Journal
• /
• v.44 no.4
• /
• pp.672-685
• /
• 2022

This article proposes a subject-independent application of brain-computer interfacing (BCI). A 32-channel Electroencephalography (EEG) device is used to measure imagined speech (SI) of four words (sos, stop, medicine, washroom) and one phrase (come-here) across 13 subjects. A deep long short-term memory (LSTM) network has been adopted to recognize the above signals in seven EEG frequency bands individually in nine major regions of the brain. The results show a maximum accuracy of 73.56% and a network prediction time (NPT) of 0.14 s which are superior to other state-of-the-art techniques in the literature. Our analysis reveals that the alpha band can recognize SI better than other EEG frequencies. To reinforce our findings, the above work has been compared by models based on the gated recurrent unit (GRU), convolutional neural network (CNN), and six conventional classifiers. The results show that the LSTM model has 46.86% more average accuracy in the alpha band and 74.54% less average NPT than CNN. The maximum accuracy of GRU was 8.34% less than the LSTM network. Deep networks performed better than traditional classifiers.