• Title/Summary/Keyword: long short-term memory(LSTM)

Search Result 501, Processing Time 0.032 seconds

Predicting Performance of Heavy Industry Firms in Korea with U.S. Trade Policy Data (미국 무역정책 변화가 국내 중공업 기업의 경영성과에 미치는 영향)

  • Park, Jinsoo;Kim, Kyoungho;Kim, Buomsoo;Suh, Jihae
    • The Journal of Society for e-Business Studies
    • /
    • v.22 no.4
    • /
    • pp.71-101
    • /
    • 2017
  • Since late 2016, protectionism has been a major trend in world trade with the Great Britain exiting the European Union and the United States electing Donald Trump as the 45th president. Consequently, there has been a huge public outcry regarding the negative prospects of heavy industry firms in Korea, which are highly dependent upon international trade with Western countries including the United States. In light of such trend and concerns, we have tried to predict business performance of heavy industry firms in Korea with data regarding trade policy of the United States. United States International Trade Commission (USITC) levies countervailing duties and anti-dumping duties to firms that violate its fair-trade regulations. In this study, we have performed data analysis with past records of countervailing duties and anti-dumping duties. With results from clustering analysis, it could be concluded that trade policy trends of the Unites States significantly affects the business performance of heavy industry firms in Korea. Furthermore, we have attempted to quantify such effects by employing long short-term memory (LSTM), a popular neural networks model that is well-suited to deal with sequential data. Our major contribution is that we have succeeded in empirically validating the intuitive argument and also predicting the future trend with rigorous data mining techniques. With some improvements, our results are expected to be highly relevant to designing regulations regarding heavy industry in Korea.

Sleep Deprivation Attack Detection Based on Clustering in Wireless Sensor Network (무선 센서 네트워크에서 클러스터링 기반 Sleep Deprivation Attack 탐지 모델)

  • Kim, Suk-young;Moon, Jong-sub
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.31 no.1
    • /
    • pp.83-97
    • /
    • 2021
  • Wireless sensors that make up the Wireless Sensor Network generally have extremely limited power and resources. The wireless sensor enters the sleep state at a certain interval to conserve power. The Sleep deflation attack is a deadly attack that consumes power by preventing wireless sensors from entering the sleep state, but there is no clear countermeasure. Thus, in this paper, using clustering-based binary search tree structure, the Sleep deprivation attack detection model is proposed. The model proposed in this paper utilizes one of the characteristics of both attack sensor nodes and normal sensor nodes which were classified using machine learning. The characteristics used for detection were determined using Long Short-Term Memory, Decision Tree, Support Vector Machine, and K-Nearest Neighbor. Thresholds for judging attack sensor nodes were then learned by applying the SVM. The determined features were used in the proposed algorithm to calculate the values for attack detection, and the threshold for determining the calculated values was derived by applying SVM.Through experiments, the detection model proposed showed a detection rate of 94% when 35% of the total sensor nodes were attack sensor nodes and improvement of up to 26% in power retention.

A Data-driven Classifier for Motion Detection of Soldiers on the Battlefield using Recurrent Architectures and Hyperparameter Optimization (순환 아키텍쳐 및 하이퍼파라미터 최적화를 이용한 데이터 기반 군사 동작 판별 알고리즘)

  • Joonho Kim;Geonju Chae;Jaemin Park;Kyeong-Won Park
    • Journal of Intelligence and Information Systems
    • /
    • v.29 no.1
    • /
    • pp.107-119
    • /
    • 2023
  • The technology that recognizes a soldier's motion and movement status has recently attracted large attention as a combination of wearable technology and artificial intelligence, which is expected to upend the paradigm of troop management. The accuracy of state determination should be maintained at a high-end level to make sure of the expected vital functions both in a training situation; an evaluation and solution provision for each individual's motion, and in a combat situation; overall enhancement in managing troops. However, when input data is given as a timer series or sequence, existing feedforward networks would show overt limitations in maximizing classification performance. Since human behavior data (3-axis accelerations and 3-axis angular velocities) handled for military motion recognition requires the process of analyzing its time-dependent characteristics, this study proposes a high-performance data-driven classifier which utilizes the long-short term memory to identify the order dependence of acquired data, learning to classify eight representative military operations (Sitting, Standing, Walking, Running, Ascending, Descending, Low Crawl, and High Crawl). Since the accuracy is highly dependent on a network's learning conditions and variables, manual adjustment may neither be cost-effective nor guarantee optimal results during learning. Therefore, in this study, we optimized hyperparameters using Bayesian optimization for maximized generalization performance. As a result, the final architecture could reduce the error rate by 62.56% compared to the existing network with a similar number of learnable parameters, with the final accuracy of 98.39% for various military operations.

A study on the feasibility analysis of the current flood season: a case study of the Yongdam Dam (현행 법정홍수기 타당성 검토 및 개선에 관한 연구: 용담댐 사례)

  • Lee, Jae Hwang;Kim, Gi Joo;Kim, Young-Oh
    • Journal of Korea Water Resources Association
    • /
    • v.57 no.5
    • /
    • pp.359-369
    • /
    • 2024
  • Korea prepares for potential floods by designating June 21st to September 20th as the flood season. However, many dams in Korea have suffered from extreme floods caused by different climate patterns, as in the case of the longest consecutive rain of 54 days in the 2020's flood season. In this context, various studies have tried to develop novel methodologies to reduce flood damage, but no study has ever dealt with the validity of the current statutory flood season thus far. This study first checked the validity of the current flood season through the observation data in the 21st century and proved that the current flood season does not consider the effects of increasing precipitation trends and the changing regional rainfall characteristics. In order to deal with these limitations, this study suggested seven new alternative flood seasons in the research area. The rigid reservoir operation method (ROM) was used for reservoir simulation, and the long short-term memory (LSTM) model was used to derive predicted inflow. Finally, all alternatives were evaluated based on whether if they exceeded the design discharge of the dam and the design flood of the river. As a result, the floods in the shifted period were reduced by 0.068% and 0.33% in terms of frequency and duration, and the magnitude also decreased by 24.6%, respectively. During this period, the second evaluation method also demonstrated that flood decreased from four to two occurrences. As the result of this study, the authors expect a formal reassessment of the flood season to take place, which will ultimately lead to the preemptive flood response to changing precipitation patterns.

A Deep Learning Based Approach to Recognizing Accompanying Status of Smartphone Users Using Multimodal Data (스마트폰 다종 데이터를 활용한 딥러닝 기반의 사용자 동행 상태 인식)

  • Kim, Kilho;Choi, Sangwoo;Chae, Moon-jung;Park, Heewoong;Lee, Jaehong;Park, Jonghun
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.1
    • /
    • pp.163-177
    • /
    • 2019
  • As smartphones are getting widely used, human activity recognition (HAR) tasks for recognizing personal activities of smartphone users with multimodal data have been actively studied recently. The research area is expanding from the recognition of the simple body movement of an individual user to the recognition of low-level behavior and high-level behavior. However, HAR tasks for recognizing interaction behavior with other people, such as whether the user is accompanying or communicating with someone else, have gotten less attention so far. And previous research for recognizing interaction behavior has usually depended on audio, Bluetooth, and Wi-Fi sensors, which are vulnerable to privacy issues and require much time to collect enough data. Whereas physical sensors including accelerometer, magnetic field and gyroscope sensors are less vulnerable to privacy issues and can collect a large amount of data within a short time. In this paper, a method for detecting accompanying status based on deep learning model by only using multimodal physical sensor data, such as an accelerometer, magnetic field and gyroscope, was proposed. The accompanying status was defined as a redefinition of a part of the user interaction behavior, including whether the user is accompanying with an acquaintance at a close distance and the user is actively communicating with the acquaintance. A framework based on convolutional neural networks (CNN) and long short-term memory (LSTM) recurrent networks for classifying accompanying and conversation was proposed. First, a data preprocessing method which consists of time synchronization of multimodal data from different physical sensors, data normalization and sequence data generation was introduced. We applied the nearest interpolation to synchronize the time of collected data from different sensors. Normalization was performed for each x, y, z axis value of the sensor data, and the sequence data was generated according to the sliding window method. Then, the sequence data became the input for CNN, where feature maps representing local dependencies of the original sequence are extracted. The CNN consisted of 3 convolutional layers and did not have a pooling layer to maintain the temporal information of the sequence data. Next, LSTM recurrent networks received the feature maps, learned long-term dependencies from them and extracted features. The LSTM recurrent networks consisted of two layers, each with 128 cells. Finally, the extracted features were used for classification by softmax classifier. The loss function of the model was cross entropy function and the weights of the model were randomly initialized on a normal distribution with an average of 0 and a standard deviation of 0.1. The model was trained using adaptive moment estimation (ADAM) optimization algorithm and the mini batch size was set to 128. We applied dropout to input values of the LSTM recurrent networks to prevent overfitting. The initial learning rate was set to 0.001, and it decreased exponentially by 0.99 at the end of each epoch training. An Android smartphone application was developed and released to collect data. We collected smartphone data for a total of 18 subjects. Using the data, the model classified accompanying and conversation by 98.74% and 98.83% accuracy each. Both the F1 score and accuracy of the model were higher than the F1 score and accuracy of the majority vote classifier, support vector machine, and deep recurrent neural network. In the future research, we will focus on more rigorous multimodal sensor data synchronization methods that minimize the time stamp differences. In addition, we will further study transfer learning method that enables transfer of trained models tailored to the training data to the evaluation data that follows a different distribution. It is expected that a model capable of exhibiting robust recognition performance against changes in data that is not considered in the model learning stage will be obtained.

Comparison of Models for Stock Price Prediction Based on Keyword Search Volume According to the Social Acceptance of Artificial Intelligence (인공지능의 사회적 수용도에 따른 키워드 검색량 기반 주가예측모형 비교연구)

  • Cho, Yujung;Sohn, Kwonsang;Kwon, Ohbyung
    • Journal of Intelligence and Information Systems
    • /
    • v.27 no.1
    • /
    • pp.103-128
    • /
    • 2021
  • Recently, investors' interest and the influence of stock-related information dissemination are being considered as significant factors that explain stock returns and volume. Besides, companies that develop, distribute, or utilize innovative new technologies such as artificial intelligence have a problem that it is difficult to accurately predict a company's future stock returns and volatility due to macro-environment and market uncertainty. Market uncertainty is recognized as an obstacle to the activation and spread of artificial intelligence technology, so research is needed to mitigate this. Hence, the purpose of this study is to propose a machine learning model that predicts the volatility of a company's stock price by using the internet search volume of artificial intelligence-related technology keywords as a measure of the interest of investors. To this end, for predicting the stock market, we using the VAR(Vector Auto Regression) and deep neural network LSTM (Long Short-Term Memory). And the stock price prediction performance using keyword search volume is compared according to the technology's social acceptance stage. In addition, we also conduct the analysis of sub-technology of artificial intelligence technology to examine the change in the search volume of detailed technology keywords according to the technology acceptance stage and the effect of interest in specific technology on the stock market forecast. To this end, in this study, the words artificial intelligence, deep learning, machine learning were selected as keywords. Next, we investigated how many keywords each week appeared in online documents for five years from January 1, 2015, to December 31, 2019. The stock price and transaction volume data of KOSDAQ listed companies were also collected and used for analysis. As a result, we found that the keyword search volume for artificial intelligence technology increased as the social acceptance of artificial intelligence technology increased. In particular, starting from AlphaGo Shock, the keyword search volume for artificial intelligence itself and detailed technologies such as machine learning and deep learning appeared to increase. Also, the keyword search volume for artificial intelligence technology increases as the social acceptance stage progresses. It showed high accuracy, and it was confirmed that the acceptance stages showing the best prediction performance were different for each keyword. As a result of stock price prediction based on keyword search volume for each social acceptance stage of artificial intelligence technologies classified in this study, the awareness stage's prediction accuracy was found to be the highest. The prediction accuracy was different according to the keywords used in the stock price prediction model for each social acceptance stage. Therefore, when constructing a stock price prediction model using technology keywords, it is necessary to consider social acceptance of the technology and sub-technology classification. The results of this study provide the following implications. First, to predict the return on investment for companies based on innovative technology, it is most important to capture the recognition stage in which public interest rapidly increases in social acceptance of the technology. Second, the change in keyword search volume and the accuracy of the prediction model varies according to the social acceptance of technology should be considered in developing a Decision Support System for investment such as the big data-based Robo-advisor recently introduced by the financial sector.

Deep Learning-based Abnormal Behavior Detection System for Dementia Patients (치매 환자를 위한 딥러닝 기반 이상 행동 탐지 시스템)

  • Kim, Kookjin;Lee, Seungjin;Kim, Sungjoong;Kim, Jaegeun;Shin, Dongil;shin, Dong-kyoo
    • Journal of Internet Computing and Services
    • /
    • v.21 no.3
    • /
    • pp.133-144
    • /
    • 2020
  • The number of elderly people with dementia is increasing as fast as the proportion of older people due to aging, which creates a social and economic burden. In particular, dementia care costs, including indirect costs such as increased care costs due to lost caregiver hours and caregivers, have grown exponentially over the years. In order to reduce these costs, it is urgent to introduce a management system to care for dementia patients. Therefore, this study proposes a sensor-based abnormal behavior detection system to manage dementia patients who live alone or in an environment where they cannot always take care of dementia patients. Existing studies were merely evaluating behavior or evaluating normal behavior, and there were studies that perceived behavior by processing images, not data from sensors. In this study, we recognized the limitation of real data collection and used both the auto-encoder, the unsupervised learning model, and the LSTM, the supervised learning model. Autoencoder, an unsupervised learning model, trained normal behavioral data to learn patterns for normal behavior, and LSTM further refined classification by learning behaviors that could be perceived by sensors. The test results show that each model has about 96% and 98% accuracy and is designed to pass the LSTM model when the autoencoder outlier has more than 3%. The system is expected to effectively manage the elderly and dementia patients who live alone and reduce the cost of caring.

A Study on Deep Learning Model for Discrimination of Illegal Financial Advertisements on the Internet

  • Kil-Sang Yoo; Jin-Hee Jang;Seong-Ju Kim;Kwang-Yong Gim
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.8
    • /
    • pp.21-30
    • /
    • 2023
  • The study proposes a model that utilizes Python-based deep learning text classification techniques to detect the legality of illegal financial advertising posts on the internet. These posts aim to promote unlawful financial activities, including the trading of bank accounts, credit card fraud, cashing out through mobile payments, and the sale of personal credit information. Despite the efforts of financial regulatory authorities, the prevalence of illegal financial activities persists. By applying this proposed model, the intention is to aid in identifying and detecting illicit content in internet-based illegal financial advertisining, thus contributing to the ongoing efforts to combat such activities. The study utilizes convolutional neural networks(CNN) and recurrent neural networks(RNN, LSTM, GRU), which are commonly used text classification techniques. The raw data for the model is based on manually confirmed regulatory judgments. By adjusting the hyperparameters of the Korean natural language processing and deep learning models, the study has achieved an optimized model with the best performance. This research holds significant meaning as it presents a deep learning model for discerning internet illegal financial advertising, which has not been previously explored. Additionally, with an accuracy range of 91.3% to 93.4% in a deep learning model, there is a hopeful anticipation for the practical application of this model in the task of detecting illicit financial advertisements, ultimately contributing to the eradication of such unlawful financial advertisements.

Water temperature prediction of Daecheong Reservoir by a process-guided deep learning model (역학적 모델과 딥러닝 모델을 융합한 대청호 수온 예측)

  • Kim, Sung Jin;Park, Hyungseok;Lee, Gun Ho;Chung, Se Woong
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2021.06a
    • /
    • pp.88-88
    • /
    • 2021
  • 최근 수자원과 수질관리 분야에 자료기반 머신러닝 모델과 딥러닝 모델의 활용이 급증하고 있다. 그러나 딥러닝 모델은 Blackbox 모델의 특성상 고전적인 질량, 운동량, 에너지 보존법칙을 고려하지 않고, 데이터에 내재된 패턴과 관계를 해석하기 때문에 물리적 법칙을 만족하지 않는 예측결과를 가져올 수 있다. 또한, 딥러닝 모델의 예측 성능은 학습데이터의 양과 변수 선정에 크게 영향을 받는 모델이기 때문에 양질의 데이터가 제공되지 않으면 모델의 bias와 variation이 클 수 있으며 정확도 높은 예측이 어렵다. 최근 이러한 자료기반 모델링 방법의 단점을 보완하기 위해 프로세스 기반 수치모델과 딥러닝 모델을 결합하여 두 모델링 방법의 장점을 활용하는 연구가 활발히 진행되고 있다(Read et al., 2019). Process-Guided Deep Learning (PGDL) 방법은 물리적 법칙을 반영하여 딥러닝 모델을 훈련시킴으로써 순수한 딥러닝 모델의 물리적 법칙 결여성 문제를 해결할 수 있는 대안으로 활용되고 있다. PGDL 모델은 딥러닝 모델에 물리적인 법칙을 해석할 수 있는 추가변수를 도입하며, 딥러닝 모델의 매개변수 최적화 과정에서 Cost 함수에 물리적 법칙을 위반하는 경우 Penalty를 추가하는 알고리즘을 도입하여 물리적 보존법칙을 만족하도록 모델을 훈련시킨다. 본 연구의 목적은 대청호의 수심별 수온을 예측하기 위해 역학적 모델과 딥러닝 모델을 융합한 PGDL 모델을 개발하고 적용성을 평가하는데 있다. 역학적 모델은 2차원 횡방향 평균 수리·수질 모델인 CE-QUAL-W2을 사용하였으며, 대청호를 대상으로 2017년부터 2018년까지 총 2년간 수온과 에너지 수지를 모의하였다. 기상(기온, 이슬점온도, 풍향, 풍속, 운량), 수문(저수위, 유입·유출 유량), 수온자료를 수집하여 CE-QUAL-W2 모델을 구축하고 보정하였으며, 모델은 저수위 변화, 수온의 수심별 시계열 변동 특성을 적절하게 재현하였다. 또한, 동일기간 대청호 수심별 수온 예측을 위한 순환 신경망 모델인 LSTM(Long Short-Term Memory)을 개발하였으며, 종속변수는 수온계 체인을 통해 수집한 수심별 고빈도 수온 자료를 사용하고 독립 변수는 기온, 풍속, 상대습도, 강수량, 단파복사에너지, 장파복사에너지를 사용하였다. LSTM 모델의 매개변수 최적화는 지도학습을 통해 예측값과 실측값의 RMSE가 최소화 되로록 훈련하였다. PGDL 모델은 동일 기간 LSTM 모델과 동일 입력 자료를 사용하여 구축하였으며, 역학적 모델에서 얻은 에너지 수지를 만족하지 않는 경우 Cost Function에 Penalty를 추가하여 물리적 보존법칙을 만족하도록 훈련하고 수심별 수온 예측결과를 비교·분석하였다.

  • PDF

Improving Bidirectional LSTM-CRF model Of Sequence Tagging by using Ontology knowledge based feature (온톨로지 지식 기반 특성치를 활용한 Bidirectional LSTM-CRF 모델의 시퀀스 태깅 성능 향상에 관한 연구)

  • Jin, Seunghee;Jang, Heewon;Kim, Wooju
    • Journal of Intelligence and Information Systems
    • /
    • v.24 no.1
    • /
    • pp.253-266
    • /
    • 2018
  • This paper proposes a methodology applying sequence tagging methodology to improve the performance of NER(Named Entity Recognition) used in QA system. In order to retrieve the correct answers stored in the database, it is necessary to switch the user's query into a language of the database such as SQL(Structured Query Language). Then, the computer can recognize the language of the user. This is the process of identifying the class or data name contained in the database. The method of retrieving the words contained in the query in the existing database and recognizing the object does not identify the homophone and the word phrases because it does not consider the context of the user's query. If there are multiple search results, all of them are returned as a result, so there can be many interpretations on the query and the time complexity for the calculation becomes large. To overcome these, this study aims to solve this problem by reflecting the contextual meaning of the query using Bidirectional LSTM-CRF. Also we tried to solve the disadvantages of the neural network model which can't identify the untrained words by using ontology knowledge based feature. Experiments were conducted on the ontology knowledge base of music domain and the performance was evaluated. In order to accurately evaluate the performance of the L-Bidirectional LSTM-CRF proposed in this study, we experimented with converting the words included in the learned query into untrained words in order to test whether the words were included in the database but correctly identified the untrained words. As a result, it was possible to recognize objects considering the context and can recognize the untrained words without re-training the L-Bidirectional LSTM-CRF mode, and it is confirmed that the performance of the object recognition as a whole is improved.