• KIISE Transactions on Computing Practices
• /
• v.22 no.7
• /
• pp.315-319
• /
• 2016

Recurrent neural network based language models (RNN LM) have shown improved results in language model researches. The RNN LMs are limited to post processing sessions, such as the N-best rescoring step of the wFST based speech recognition. However, it has considerable vocabulary problems that require large computing powers for the LM training. In this paper, we try to find the 1st pass N-gram model using word embedding, which is the simplified deep neural network. The class based language model (LM) can be a way to approach to this issue. We have built class based vocabulary through word embedding, by combining the class LM with word N-gram LM to evaluate the performance of LMs. In addition, we propose that part-of-speech (POS) tagging based LM shows an improvement of perplexity in all types of the LM tests.

• Journal of The Korean Society of Agricultural Engineers
• /
• v.61 no.6
• /
• pp.123-132
• /
• 2019

This study is to estimate the spatial soil moisture using Terra MODIS (Moderate Resolution Imaging Spectroradiometer) satellite data and machine learning technique. Using the 3 years (2015~2017) data of MODIS 16 days composite NDVI (Normalized Difference Vegetation Index) and daily Land Surface Temperature (LST), ground measured precipitation and sunshine hour of KMA (Korea Meteorological Administration), the RDA (Rural Development Administration) 10 cm~30 cm average TDR (Time Domain Reflectometry) measured soil moisture at 78 locations was tested. For daily analysis, the missing values of MODIS LST by clouds were interpolated by conditional merging method using KMA surface temperature observation data, and the 16 days NDVI was linearly interpolated to 1 day interval. By applying the RNN-LSTM (Recurrent Neural Network-Long Short Term Memory) artificial neural network model, 70% of the total period was trained and the rest 30% period was verified. The results showed that the coefficient of determination ($R^2$), Root Mean Square Error (RMSE), and Nash-Sutcliffe Efficiency were 0.78, 2.76%, and 0.75 respectively. In average, the clay soil moisture was estimated well comparing with the other soil types of silt, loam, and sand. This is because the clay has the intrinsic physical property for having narrow range of soil moisture variation between field capacity and wilting point.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.3
• /
• pp.837-852
• /
• 2021

Recurrent neural network (RNN) architectures have been used for language modeling (LM) tasks that require learning long-range word or character sequences. However, the RNN architecture is still suffered from unstable gradients on long-range sequences. To address the issue of long-range sequences, an attention mechanism has been used, showing state-of-the-art (SOTA) performance in all LM tasks. A differentiable neural computer (DNC) is a deep learning architecture using an attention mechanism. The DNC architecture is a neural network augmented with a content-addressable external memory. However, in the write operation, some information unrelated to the input word remains in memory. Moreover, DNCs have been found to perform poorly with low numbers of weight parameters. Therefore, we propose a robust memory deallocation method using a limited retention vector. The limited retention vector determines whether the network increases or decreases its usage of information in external memory according to a threshold. We experimentally evaluate the robustness of a DNC implementing the proposed approach according to the size of the controller and external memory on the enwik8 LM task. When we decreased the number of weight parameters by 32.47%, the proposed DNC showed a low bits-per-character (BPC) degradation of 4.30%, demonstrating the effectiveness of our approach in language modeling tasks.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2022.10a
• /
• pp.201-203
• /
• 2022

Over the past five years, the number of patients treated for diabetes has increased by 27.7% to 3.22 million, and since blood sugar is still checked through finger blood collection, continuous blood glucose measurement and blood sugar peak confirmation are difficult and painful. To solve this problem, based on blood sugar data measured for 14 days, three months of blood sugar prediction data are provided to diabetics using artificial intelligence technology.

• Phonetics and Speech Sciences
• /
• v.11 no.2
• /
• pp.57-64
• /
• 2019

In this paper, we construct a Korean text-to-speech system using the Merlin toolkit which is an open source system for speech synthesis. In the text-to-speech system, the HMM-based statistical parametric speech synthesis method is widely used, but it is known that the quality of synthesized speech is degraded due to limitations of the acoustic modeling scheme that includes context factors. In this paper, we propose an acoustic modeling architecture that uses deep neural network technique, which shows excellent performance in various fields. Fully connected deep feedforward neural network (DNN), recurrent neural network (RNN), gated recurrent unit (GRU), long short-term memory (LSTM), bidirectional LSTM (BLSTM) are included in the architecture. Experimental results have shown that the performance is improved by including sequence modeling in the architecture, and the architecture with LSTM or BLSTM shows the best performance. It has been also found that inclusion of delta and delta-delta components in the acoustic feature parameters is advantageous for performance improvement.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2020.06a
• /
• pp.219-219
• /
• 2020

하천의 수질을 나타내는 환경지표 중 국가 TMS(Tele Monitoring system)의 수질측정망을 통해 관리되고 있는 지표로는 DO, BOD, COD, SS, TN, TP 등 여러 인자들이 있다. 이러한 수질인자는 하천의 자정작용에 있어 많은 영향을 나타내고 있다. 이를 활용한 경제적이고 합리적인 수질관리를 위해 하천의 자정작용을 활용하는 것이 중요하다. 생물학적 작용을 가장 효과적으로 활용하기 위해서는 수질오염 데이터에 기초한 수질예측을 채택하여 적절한 대책이 필요하다. 이를 위해서는 수질인자의 데이터를 측정하고 축적해 수질오염을 예측하는 것이 필수적인데, 실제적으로 수질인자의 일일 측정은 비용 관점에서 쉽게 접근할 수 없다. 본 연구에서는 시계열 학습으로 알려진 RNN-LSTM(Recurrent Neural Network-Long Term Memory) 알고리즘을 활용하여 기존에 측정된 수질인자의 데이터를 통해 시간당 및 일일 수질인자를 예측하려고 했다. 연구에 앞서, 기존에 시간단위로 측정된 수질인자 데이터의 이상 유무를 확인 후, 에러값은 제거하고 12시간 이하 데이터가 누락되었을 때는 선형 보간하여 데이터를 사용하고, 1일 데이터도 10일 이하 데이터가 누락되었을 때 선형 보간하여 데이터를 활용하여 수질인자를 예측하였다. 수질인자를 예측하기 위해 구글이 개발한 딥러닝 오픈소스 라이브러리인 텐서플로우를 활용하였고, 연구지역으로는 대한민국 부산에 위치한 온천천의 유역을 선정하였다. 수질인자 데이터 수집은 부산광역시에서 운영하는 보건환경정보 공개시스템의 자료를 활용하였다. 모델의 연구를 위해 하천의 수질인자, 기상자료 데이터를 입력자료로 활용하였다. 분석에서는 입력자료와, 반복횟수, 시계열의 길이 등을 조절해 수질 요인을 예측했고, 모델의 정확도도 분석하였다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.13 no.4
• /
• pp.2060-2077
• /
• 2019

Recently, mobile healthcare services have attracted significant attention because of the emerging development and supply of diverse wearable devices. Smartwatches and health bands are the most common type of mobile-based wearable devices and their market size is increasing considerably. However, simple value comparisons based on accumulated data have revealed certain problems, such as the standardized nature of health management and the lack of personalized health management service models. The convergence of information technology (IT) and biotechnology (BT) has shifted the medical paradigm from continuous health management and disease prevention to the development of a system that can be used to provide ground-based medical services regardless of the user's location. Moreover, the IT-BT convergence has necessitated the development of lifestyle improvement models and services that utilize big data analysis and machine learning to provide mobile healthcare-based personal health management and disease prevention information. Users' health data, which are specific as they change over time, are collected by different means according to the users' lifestyle and surrounding circumstances. In this paper, we propose a prediction model of user physical activity that uses data characteristics-based long short-term memory (DC-LSTM) recurrent neural networks (RNNs). To provide personalized services, the characteristics and surrounding circumstances of data collectable from mobile host devices were considered in the selection of variables for the model. The data characteristics considered were ease of collection, which represents whether or not variables are collectable, and frequency of occurrence, which represents whether or not changes made to input values constitute significant variables in terms of activity. The variables selected for providing personalized services were activity, weather, temperature, mean daily temperature, humidity, UV, fine dust, asthma and lung disease probability index, skin disease probability index, cadence, travel distance, mean heart rate, and sleep hours. The selected variables were classified according to the data characteristics. To predict activity, an LSTM RNN was built that uses the classified variables as input data and learns the dynamic characteristics of time series data. LSTM RNNs resolve the vanishing gradient problem that occurs in existing RNNs. They are classified into three different types according to data characteristics and constructed through connections among the LSTMs. The constructed neural network learns training data and predicts user activity. To evaluate the proposed model, the root mean square error (RMSE) was used in the performance evaluation of the user physical activity prediction method for which an autoregressive integrated moving average (ARIMA) model, a convolutional neural network (CNN), and an RNN were used. The results show that the proposed DC-LSTM RNN method yields an excellent mean RMSE value of 0.616. The proposed method is used for predicting significant activity considering the surrounding circumstances and user status utilizing the existing standardized activity prediction services. It can also be used to predict user physical activity and provide personalized healthcare based on the data collectable from mobile host devices.

• Korean Journal of Agricultural Science
• /
• v.46 no.4
• /
• pp.843-856
• /
• 2019

Real-time flood prediction has an important role in significantly reducing potential damage caused by floods for urban residential areas located downstream of river basins. This paper presents an effective approach for flood forecasting based on the construction of a deep neural network (DNN) model. In addition, this research depends closely on the open-source software library, TensorFlow, which was developed by Google for machine and deep learning applications and research. The proposed model was applied to forecast the flowrate one, two, and three days in advance at the Son Tay hydrological station on the Red River, Vietnam. The input data of the model was a series of discharge data observed at five gauge stations on the Red River system, without requiring rainfall data, water levels and topographic characteristics. The research results indicate that the DNN model achieved a high performance for flood forecasting even though only a modest amount of data is required. When forecasting one and two days in advance, the Nash-Sutcliffe Efficiency (NSE) reached 0.993 and 0.938, respectively. The findings of this study suggest that the DNN model can be used to construct a real-time flood warning system on the Red River and for other river basins in Vietnam.

• Proceedings of the KIEE Conference
• /
• 1997.07b
• /
• pp.400-402
• /
• 1997

This paper investigates the direct and adaptive control of mechatronic servo systems using modified chaotic neural networks (CNNs). For the performance evaluation of the proposed neural networks, we simulate the trajectory control of the X-Y table with direct control strategies. The CNN based controller demonstrates accurate tracking of the planned path and also shows superior performance on convergence and final error comparing with recurrent neural network(RNN) controller.

• Journal of the Korea Society of Computer and Information
• /
• v.25 no.9
• /
• pp.37-44
• /
• 2020

AI technology has developed in the form of decision support technology in law, patent, finance and national defense and is applied to disease diagnosis and legal judgment. To search real-time information with Deep Learning, Big data Analysis and Deep Learning Algorithm are required. In this paper, we try to predict the entrance rate to high-ranking universities using a Deep Learning model, RNN(Recurrent Neural Network). First, we analyzed the current status of private academies in administrative districts and the number of students by age in administrative districts, and established a socially accepted hypothesis that students residing in areas with a high educational fever have a high rate of enrollment in high-ranking universities. This is to verify based on the data analyzed using the predicted hypothesis and the government's public data. The predictive model uses data from 2015 to 2017 to learn to predict the top enrollment rate, and the trained model predicts the top enrollment rate in 2018. A prediction experiment was performed using RNN, a Deep Learning model, for the high-ranking enrollment rate in the special education zone. In this paper, we define the correlation between the high-ranking enrollment rate by analyzing the household income and the participation rate of private education about the current status of private institutes in regions with high education fever and the effect on the number of students by age.