• Korean Journal of Mathematics
• /
• v.30 no.1
• /
• pp.131-146
• /
• 2022

This is an expository article about the gradients in deep neural network. It is hard to find a place where gradients in a deep neural network are dealt in details in a systematic and mathematical way. We review and compute the gradients and Jacobians to derive formulas for gradients which appear in the backpropagation and implement them in vectorized forms in Python.

• Geophysics and Geophysical Exploration
• /
• v.24 no.2
• /
• pp.53-66
• /
• 2021

Velocity model building is an essential procedure in seismic data processing. Conventional techniques, such as traveltime tomography or velocity analysis take longer computational time to predict a single velocity model and the quality of the inversion results is highly dependent on human expertise. Full-waveform inversions also depend on an accurate initial model. Recently, deep neural network techniques are gaining widespread acceptance due to an increase in their integration to solving complex and nonlinear problems. This study investigated cases of seismic velocity model building using deep neural network techniques by classifying items according to the neural networks used in each study. We also included cases of generating training synthetic velocity models. Deep neural networks automatically optimize model parameters by training neural networks from large amounts of data. Thus, less human interaction is involved in the quality of the inversion results compared to that of conventional techniques and the computational cost of predicting a single velocity model after training is negligible. Additionally, unlike full-waveform inversions, the initial velocity model is not required. Several studies have demonstrated that deep neural network techniques achieve outstanding performance not only in computational cost but also in inversion results. Based on the research results, we analyzed and discussed the characteristics of deep neural network techniques for building velocity models.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.22 no.5
• /
• pp.728-733
• /
• 2018

Deep learning has been actively studied for predicting protein secondary structure based only on the sequence information of the amino acids constituting the protein. In this paper, we compared the performances of the convolutional neural networks of various structures to predict the protein secondary structure. To investigate the optimal depth of the layer of neural network for the prediction of protein secondary structure, the performance according to the number of layers was investigated. We also applied the structure of GoogLeNet and ResNet which constitute building blocks of many image classification methods. These methods extract various features from input data, and smooth the gradient transmission in the learning process even using the deep layer. These architectures of convolutional neural networks were modified to suit the characteristics of protein data to improve performance.

• Database Research
• /
• v.34 no.3
• /
• pp.3-13
• /
• 2018

Text classification is one of the text mining technologies that classifies a given textual document into its appropriate categories and is used in various fields such as spam email detection, news classification, question answering, emotional analysis, and chat bot. In general, the text classification system utilizes machine learning algorithms, and among a number of algorithms, naïve Bayes and support vector machine, which are suitable for text data, are known to have reasonable performance. Recently, with the development of deep learning technology, several researches on applying deep neural networks such as recurrent neural networks (RNN) and convolutional neural networks (CNN) have been introduced to improve the performance of text classification system. However, the current text classification techniques have not yet reached the perfect level of text classification. This paper focuses on the fact that the text data is expressed as a vector only with the word dimensions, which impairs the semantic information inherent in the text, and proposes a neural network architecture based upon the semantic tensor space model.

• KIPS Transactions on Software and Data Engineering
• /
• v.7 no.8
• /
• pp.307-324
• /
• 2018

Recently, recurrent neural networks have been attracting attention in solving prediction problem of sequential data through structure considering time dependency. However, as the time step of sequential data increases, the problem of the gradient vanishing is occurred. Long short-term memory models have been proposed to solve this problem, but there is a limit to storing a lot of data and preserving it for a long time. Therefore, research on memory-augmented neural network (MANN), which is a learning model using recurrent neural networks and memory elements, has been actively conducted. In this paper, we describe the structure and characteristics of MANN models that emerged as a hot topic in deep learning field and present the latest techniques and future research that utilize MANN.

• Smart Media Journal
• /
• v.11 no.3
• /
• pp.26-30
• /
• 2022

Spiking neural network is a neural network that applies the working principle of real brain neurons. Due to the biological mechanism of neurons, it consumes less power for training and reasoning than conventional neural networks. Recently, as deep learning models become huge and operating costs increase exponentially, the spiking neural network is attracting attention as a third-generation neural network that connects convolution neural networks and recurrent neural networks, and related research is being actively conducted. However, in order to apply the spiking neural network model to the industry, a lot of research still needs to be done, and the problem of model retraining to apply a new model must also be solved. In this paper, we propose a method to minimize the cost of model retraining by extracting the weights of the existing trained deep learning model and converting them into the weights of the spiking neural network model. In addition, it was found that weight conversion worked correctly by comparing the results of inference using the converted weights with the results of the existing model.

• IEIE Transactions on Smart Processing and Computing
• /
• v.4 no.1
• /
• pp.35-43
• /
• 2015

Over the past couple of years, tremendous progress has been made in applying deep learning (DL) techniques to computer vision. Especially, deep convolutional neural networks (DCNNs) have achieved state-of-the-art performance on standard recognition datasets and tasks such as ImageNet Large-Scale Visual Recognition Challenge (ILSVRC). Among them, GoogLeNet network which is a radically redesigned DCNN based on the Hebbian principle and scale invariance set the new state of the art for classification and detection in the ILSVRC 2014. Since there exist various deep learning techniques, this review paper is focusing on techniques directly related to DCNNs, especially those needed to understand the architecture and techniques employed in GoogLeNet network.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.12 no.10
• /
• pp.5159-5178
• /
• 2018

Network Intrusion detection is a rapidly growing field of information security due to its importance for modern IT infrastructure. Many supervised and unsupervised learning techniques have been devised by researchers from discipline of machine learning and data mining to achieve reliable detection of anomalies. In this paper, a deep convolutional neural network (DCNN) based intrusion detection system (IDS) is proposed, implemented and analyzed. Deep CNN core of proposed IDS is fine-tuned using Randomized search over configuration space. Proposed system is trained and tested on NSLKDD training and testing datasets using GPU. Performance comparisons of proposed DCNN model are provided with other classifiers using well-known metrics including Receiver operating characteristics (RoC) curve, Area under RoC curve (AuC), accuracy, precision-recall curve and mean average precision (mAP). The experimental results of proposed DCNN based IDS shows promising results for real world application in anomaly detection systems.

• Proceedings of the Korean Institute of Navigation and Port Research Conference
• /
• 2022.11a
• /
• pp.383-384
• /
• 2022

Gas Combustion Unit (GCU) onboard liquefied natural gas carriers handles boil-off to stabilize tank pressure. There are many factors for LNG cargo operators to take into consideration to determine whether to use GCU or not. Gas consumption of main engine and re-liquefied gas through the Partial Re-Liquefaction System (PRS) are good examples of these factors. Human gas operators have decided the operation so far. In this paper, some deep learning neural network models were developed to provide human gas operators with a decision support system. The models consider various factors specially into GCU operation. A deep learning model with Sigmoid activation functions in input layer and hidden layers made the best performance among eight different deep learning models.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.25 no.5
• /
• pp.425-430
• /
• 2015

This paper is a study to improve the classification efficiency of rotating objects by using deep neural networks to which a deep learning algorithm was applied. For the classification experiment of rotating objects, COIL-20 is used as data and total 3 types of classifiers are compared and analyzed. 3 types of classifiers used in the study include PCA classifier to derive a feature value while reducing the dimension of data by using Principal Component Analysis and classify by using euclidean distance, MLP classifier of the way of reducing the error energy by using error back-propagation algorithm and finally, deep learning applied DBN classifier of the way of increasing the probability of observing learning data through pre-training and reducing the error energy through fine-tuning. In order to identify the structure-specific error rate of the deep neural networks, the experiment is carried out while changing the number of hidden layers and number of hidden neurons. The classifier using DBN showed the lowest error rate. Its structure of deep neural networks with 2 hidden layers showed a high recognition rate by moving parameters to a location helpful for recognition.