• Proceedings of the Korea Water Resources Association Conference
• /
• 2021.06a
• /
• pp.292-292
• /
• 2021

최근 데이터 과학의 획기적인 발전으로 딥러닝(Deep Learning) 알고리즘이 개발되어 다양한 분야에 널리 적용되고 있다. 본 연구에서는 인공신경망 중 하나인 LSTM(Long-Short Term Memory) 네트워크를 기반으로 정확한 댐유입량 예측을 수행하는 SWLSTM 모델을 제안하였다. SWLSM은 모델의 정확도를 개선하기 위해 세 가지 주요 아이디어를 채택하였다. (1) 통계적 속성 (PACF) 및 교차 상관 함수(CCF)를 사용하여 적절한 입력 변수와 시퀀스 길이를 결정하였다. (2) 선택된 입력 예측 변수 시계열을 웨이블릿 변환(WT)을 사용하여 하위 시계열로 분해한다. (3) k-folds cross validation 및 random search 기법을 사용하여 LSTM의 하이퍼 매개변수들을 효율적으로 최적화하고 검증한다. 제안된 SWLSTM의 효과는 한강 유역 5개 댐의 시단위/일단위/월단위 유입량을 예측하고 과거 자료와 비교함으로써 검증하였다. 모델의 정확도는 다양한 평가 메트릭(R2, NSE, MAE, PE)이 사용하였으며, SWLSTM은 모든 경우에서 LSTM 모델을 능가하였다. (평가 지표는 약 30 ~ 80 % 더 나은 성능을 보여줌). 본 연구의 결과로부터, 올바른 입력 변수와 시퀀스 길이의 선택이 모델 학습의 효율성을 높이고 노이즈를 줄이는 데 효과적임을 확인하였다. WT는 홍수 첨두와 같은 극단적인 값을 예측하는 데 도움이 된다. k-folds cross validation 및 random search 기법을 사용하면 모델의 하이퍼 매개변수를 효율적으로 설정할 수 있다. 본 연구로부터 댐 유입량을 정확하게 예측한다면 정책 입안자와 운영자가 저수지 운영, 계획 및 관리에 도움이 될 것이다.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2022.05a
• /
• pp.348-348
• /
• 2022

미래에 대한 기후는 과거와 비교하여 변동성이 더 크고 불확실성 또한 더 크기 때문에 미래의 기후변화를 예측하기 위해서는 기후변화의 절대적인 크기뿐 아니라 불확실한 정도도 함께 고려되어야 한다. 본 연구에서는 CMIP6(Coupled Model Intercomparison Project Phase 6) DB에서 제공된 일 단위 18개의 GCMs(General Circulation Models)의 결과를 분석하였으며 또한 3개의SSP(Shared Socioeconomic Pathway)시나리오와 3개의 미래 구간에 대하여 100개의 앙상블을 각각 생성하였다. 불확실성을 초래하는 원인을 3가지로 구분하고, 각각의 원인에 대한 불확실성의 정도를 앙상블 시나리오에 반영하고자 한다. 현재 기간 및 미래 기간에 대해 100개의 20년 시계열 날씨변수 앙상블을 생성하여 LSTM(Long short-term memory)의 입력자료로 사용하여 댐유입량, 저수위, 방류량을 산정하였다. 댐 유입량 및 방류량의 예측성능을 향상시키기 위해 Input predictor의 종류를 선정하는 방법과 그 변수들의 lag time을 결정하는 방법, 입력자료들을 재구성하는 방법, 하이퍼 매개변수를 효율적으로 최적화하는 방법, 목적함수 설정 방법들을 제시하여 댐 유입량 및 방류량의 예측을 크게 향상시키고자 하였다. 본 연구에서 예측된 미래의 댐유입량 및 방류량 정보는 홍수 또는 가뭄 등 다양한 수자원 관련 문제의 전략을 수립하는 데 있어서 적절한 도움이 될 것이다.

• Journal of Intelligence and Information Systems
• /
• v.25 no.4
• /
• pp.141-154
• /
• 2019

Rapid growth of internet technology and social media is progressing. Data mining technology has evolved to enable unstructured document representations in a variety of applications. Sentiment analysis is an important technology that can distinguish poor or high-quality content through text data of products, and it has proliferated during text mining. Sentiment analysis mainly analyzes people's opinions in text data by assigning predefined data categories as positive and negative. This has been studied in various directions in terms of accuracy from simple rule-based to dictionary-based approaches using predefined labels. In fact, sentiment analysis is one of the most active researches in natural language processing and is widely studied in text mining. When real online reviews aren't available for others, it's not only easy to openly collect information, but it also affects your business. In marketing, real-world information from customers is gathered on websites, not surveys. Depending on whether the website's posts are positive or negative, the customer response is reflected in the sales and tries to identify the information. However, many reviews on a website are not always good, and difficult to identify. The earlier studies in this research area used the reviews data of the Amazon.com shopping mal, but the research data used in the recent studies uses the data for stock market trends, blogs, news articles, weather forecasts, IMDB, and facebook etc. However, the lack of accuracy is recognized because sentiment calculations are changed according to the subject, paragraph, sentiment lexicon direction, and sentence strength. This study aims to classify the polarity analysis of sentiment analysis into positive and negative categories and increase the prediction accuracy of the polarity analysis using the pretrained IMDB review data set. First, the text classification algorithm related to sentiment analysis adopts the popular machine learning algorithms such as NB (naive bayes), SVM (support vector machines), XGboost, RF (random forests), and Gradient Boost as comparative models. Second, deep learning has demonstrated discriminative features that can extract complex features of data. Representative algorithms are CNN (convolution neural networks), RNN (recurrent neural networks), LSTM (long-short term memory). CNN can be used similarly to BoW when processing a sentence in vector format, but does not consider sequential data attributes. RNN can handle well in order because it takes into account the time information of the data, but there is a long-term dependency on memory. To solve the problem of long-term dependence, LSTM is used. For the comparison, CNN and LSTM were chosen as simple deep learning models. In addition to classical machine learning algorithms, CNN, LSTM, and the integrated models were analyzed. Although there are many parameters for the algorithms, we examined the relationship between numerical value and precision to find the optimal combination. And, we tried to figure out how the models work well for sentiment analysis and how these models work. This study proposes integrated CNN and LSTM algorithms to extract the positive and negative features of text analysis. The reasons for mixing these two algorithms are as follows. CNN can extract features for the classification automatically by applying convolution layer and massively parallel processing. LSTM is not capable of highly parallel processing. Like faucets, the LSTM has input, output, and forget gates that can be moved and controlled at a desired time. These gates have the advantage of placing memory blocks on hidden nodes. The memory block of the LSTM may not store all the data, but it can solve the CNN's long-term dependency problem. Furthermore, when LSTM is used in CNN's pooling layer, it has an end-to-end structure, so that spatial and temporal features can be designed simultaneously. In combination with CNN-LSTM, 90.33% accuracy was measured. This is slower than CNN, but faster than LSTM. The presented model was more accurate than other models. In addition, each word embedding layer can be improved when training the kernel step by step. CNN-LSTM can improve the weakness of each model, and there is an advantage of improving the learning by layer using the end-to-end structure of LSTM. Based on these reasons, this study tries to enhance the classification accuracy of movie reviews using the integrated CNN-LSTM model.

• International Journal of Computer Science & Network Security
• /
• v.23 no.8
• /
• pp.177-189
• /
• 2023

Malware detection is an increasingly important operational focus in cyber security, particularly given the fast pace of such threats (e.g., new malware variants introduced every day). There has been great interest in exploring the use of machine learning techniques in automating and enhancing the effectiveness of malware detection and analysis. In this paper, we present a deep recurrent neural network solution as a stacked Long Short-Term Memory (LSTM) with a pre-training as a regularization method to avoid random network initialization. In our proposal, we use global and short dependencies of the inputs. With pre-training, we avoid random initialization and are able to improve the accuracy and robustness of malware threat hunting. The proposed method speeds up the convergence (in comparison to stacked LSTM) by reducing the length of malware OpCode or bytecode sequences. Hence, the complexity of our final method is reduced. This leads to better accuracy, higher Mattews Correlation Coefficients (MCC), and Area Under the Curve (AUC) in comparison to a standard LSTM with similar detection time. Our proposed method can be applied in real-time malware threat hunting, particularly for safety critical systems such as eHealth or Internet of Military of Things where poor convergence of the model could lead to catastrophic consequences. We evaluate the effectiveness of our proposed method on Windows, Ransomware, Internet of Things (IoT), and Android malware datasets using both static and dynamic analysis. For the IoT malware detection, we also present a comparative summary of the performance on an IoT-specific dataset of our proposed method and the standard stacked LSTM method. More specifically, of our proposed method achieves an accuracy of 99.1% in detecting IoT malware samples, with AUC of 0.985, and MCC of 0.95; thus, outperforming standard LSTM based methods in these key metrics.

• KIISE Transactions on Computing Practices
• /
• v.23 no.5
• /
• pp.322-327
• /
• 2017

The classification problem in the field of Natural Language Processing has been studied for a long time. Continuing forward with our previous research, which classifies large-scale text using Convolutional Neural Networks (CNN), we implemented Recurrent Neural Networks (RNN), Long-Short Term Memory (LSTM) and Gated Recurrent Units (GRU). The experiment's result revealed that the performance of classification algorithms was Multinomial Naïve Bayesian Classifier < Support Vector Machine (SVM) < LSTM < CNN < GRU, in order. The result can be interpreted as follows: First, the result of CNN was better than LSTM. Therefore, the text classification problem might be related more to feature extraction problem than to natural language understanding problems. Second, judging from the results the GRU showed better performance in feature extraction than LSTM. Finally, the result that the GRU was better than CNN implies that text classification algorithms should consider feature extraction and sequential information. We presented the results of fine-tuning in deep neural networks to provide some intuition regard natural language processing to future researchers.

• Journal of the Korea Convergence Society
• /
• v.8 no.12
• /
• pp.55-62
• /
• 2017

The Named Entity Recognition system is a system that recognizes words or phrases with object names such as personal name (PS), place name (LC), and group name (OG) in the document as corresponding object names. Traditional approaches to named entity recognition include statistical-based models that learn models based on hand-crafted features. Recently, it has been proposed to construct the qualities expressing the sentence using models such as deep-learning based Recurrent Neural Networks (RNN) and long-short term memory (LSTM) to solve the problem of sequence labeling. In this research, to improve the performance of the Korean named entity recognition system, we used a hand-crafted feature, part-of-speech tagging information, and pre-built lexicon information to augment features for representing sentence. Experimental results show that the proposed method improves the performance of Korean named entity recognition system. The results of this study are presented through github for future collaborative research with researchers studying Korean Natural Language Processing (NLP) and named entity recognition system.

• KIPS Transactions on Software and Data Engineering
• /
• v.10 no.3
• /
• pp.91-98
• /
• 2021

Today's information and communication technology is rapidly developing, the security of IT infrastructure is becoming more important, and at the same time, cyber attacks of various forms are becoming more advanced and sophisticated like intelligent persistent attacks (Advanced Persistent Threat). Early defense or prediction of increasingly sophisticated cyber attacks is extremely important, and in many cases, the analysis of network-based intrusion detection systems (NIDS) related data alone cannot prevent rapidly changing cyber attacks. Therefore, we are currently using data generated by intrusion detection systems to protect against cyber attacks described above through Host-based Intrusion Detection System (HIDS) data analysis. In this paper, we conducted a comparative study on machine learning algorithms using LID-DS (Leipzig Intrusion Detection-Data Set) host-based intrusion detection data including thread information, metadata, and buffer data missing from previously used data sets. The algorithms used were Decision Tree, Naive Bayes, MLP (Multi-Layer Perceptron), Logistic Regression, LSTM (Long Short-Term Memory model), and RNN (Recurrent Neural Network). Accuracy, accuracy, recall, F1-Score indicators and error rates were measured for evaluation. As a result, the LSTM algorithm had the highest accuracy.

• Journal of Digital Convergence
• /
• v.19 no.8
• /
• pp.319-325
• /
• 2021

With the advancement of wireless technology and the rapid growth of the infrastructure of mobile communication technology, systems applying AI-based platforms are drawing attention from users. In particular, the system that understands users' tastes and interests and recommends preferred items is applied to advanced e-commerce customized services and smart homes. However, there is a problem that these recommendation systems are difficult to reflect in real time the preferences of various users for tastes and interests. In this research, we propose a Fuzzy-AHP-based movies recommendation system using the Gated Recurrent Unit (GRU) language model to address a problem. In this system, we apply Fuzzy-AHP to reflect users' tastes or interests in real time. We also apply GRU language model-based models to analyze the public interest and the content of the film to recommend movies similar to the user's preferred factors. To validate the performance of this recommendation system, we measured the suitability of the learning model using scraping data used in the learning module, and measured the rate of learning performance by comparing the Long Short-Term Memory (LSTM) language model with the learning time per epoch. The results show that the average cross-validation index of the learning model in this work is suitable at 94.8% and that the learning performance rate outperforms the LSTM language model.

• Journal of the Korean Society of Marine Environment & Safety
• /
• v.28 no.2
• /
• pp.193-200
• /
• 2022

We conducted a case study to determine the optimal model parameters and predictors of Long Short-Term Memory (LSTM) for the reproduction of dissolved oxygen (DO) concentration in Jinhae Bay. The model parameter case study indicated the lowest accuracy when the Hidden node=10, Epoch=100. This was caused by underfitting of machine learning. The accuracy increased as the Hidden node and Epoch increased. The accuracy was the highest when the Hidden node=80 and Epoch=100 with R²=0.99. In the bottom DO reproduction of Step 1 of the predictors case study, accuracy was highest when the water temperature was used as a predictor with R²=0.81. In Step 2, The R² value increased up to 0.92 when the water temperature and SiO₂ were used as a predictor. This was caused by a high correlation between the bottom DO and SiO₂ concentrations. Consequently, we determined the optimal model parameters and predictors of LSTM for the reproduction of DO concentration in Jinhae Bay.

• Journal of the Korea Convergence Society
• /
• v.12 no.11
• /
• pp.99-107
• /
• 2021

We live in a myriad of data. Various data are created in all situations in which we work, and we discover the meaning of data through big data technology. Many efforts are underway to find meaningful data. This paper introduces an analysis technique that enables humans to make better choices through the trend and prediction of time series data as a principal component analysis technique. Principal component analysis constructs covariance through the input data and presents eigenvectors and eigenvalues that can infer the direction of the data. The proposed method computes a reference axis in a time series data set having a similar directionality. It predicts the directionality of data in the next section through the angle between the directionality of each time series data constituting the data set and the reference axis. In this paper, we compare and verify the accuracy of the proposed algorithm with LSTM (Long Short-Term Memory) through cryptocurrency trends. As a result of comparative verification, the proposed method recorded relatively few transactions and high returns(112%) compared to LSTM in data with high volatility. It can mean that the signal was analyzed and predicted relatively accurately, and it is expected that better results can be derived through a more accurate threshold setting.