• Journal of Intelligence and Information Systems
• /
• v.22 no.2
• /
• pp.127-142
• /
• 2016

Deep learning model is a kind of neural networks that allows multiple hidden layers. There are various deep learning architectures such as convolutional neural networks, deep belief networks and recurrent neural networks. Those have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. Among those architectures, convolutional neural networks and recurrent neural networks are classified as the supervised learning model. And in recent years, those supervised learning models have gained more popularity than unsupervised learning models such as deep belief networks, because supervised learning models have shown fashionable applications in such fields mentioned above. Deep learning models can be trained with backpropagation algorithm. Backpropagation is an abbreviation for "backward propagation of errors" and a common method of training artificial neural networks used in conjunction with an optimization method such as gradient descent. The method calculates the gradient of an error function with respect to all the weights in the network. The gradient is fed to the optimization method which in turn uses it to update the weights, in an attempt to minimize the error function. Convolutional neural networks use a special architecture which is particularly well-adapted to classify images. Using this architecture makes convolutional networks fast to train. This, in turn, helps us train deep, muti-layer networks, which are very good at classifying images. These days, deep convolutional networks are used in most neural networks for image recognition. Convolutional neural networks use three basic ideas: local receptive fields, shared weights, and pooling. By local receptive fields, we mean that each neuron in the first(or any) hidden layer will be connected to a small region of the input(or previous layer's) neurons. Shared weights mean that we're going to use the same weights and bias for each of the local receptive field. This means that all the neurons in the hidden layer detect exactly the same feature, just at different locations in the input image. In addition to the convolutional layers just described, convolutional neural networks also contain pooling layers. Pooling layers are usually used immediately after convolutional layers. What the pooling layers do is to simplify the information in the output from the convolutional layer. Recent convolutional network architectures have 10 to 20 hidden layers and billions of connections between units. Training deep learning networks has taken weeks several years ago, but thanks to progress in GPU and algorithm enhancement, training time has reduced to several hours. Neural networks with time-varying behavior are known as recurrent neural networks or RNNs. A recurrent neural network is a class of artificial neural network where connections between units form a directed cycle. This creates an internal state of the network which allows it to exhibit dynamic temporal behavior. Unlike feedforward neural networks, RNNs can use their internal memory to process arbitrary sequences of inputs. Early RNN models turned out to be very difficult to train, harder even than deep feedforward networks. The reason is the unstable gradient problem such as vanishing gradient and exploding gradient. The gradient can get smaller and smaller as it is propagated back through layers. This makes learning in early layers extremely slow. The problem actually gets worse in RNNs, since gradients aren't just propagated backward through layers, they're propagated backward through time. If the network runs for a long time, that can make the gradient extremely unstable and hard to learn from. It has been possible to incorporate an idea known as long short-term memory units (LSTMs) into RNNs. LSTMs make it much easier to get good results when training RNNs, and many recent papers make use of LSTMs or related ideas.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2009.05a
• /
• pp.125-128
• /
• 2009

The research of climate change impact in hydrometeorology often relies on climate change information. In this paper, neural networks models such as generalized regression neural networks model (GRNNM) and multilayer perceptron neural networks model (MLP-NNM) are proposed statistical downscaling of the daily precipitation. The input nodes of neural networks models consist of the atmospheric meteorology and the atmospheric pressure data for 4 grid points including $127.5^{\circ}E/37.5^{\circ}N$, $127.5^{\circ}E/35^{\circ}N$, $125^{\circ}E/37.5^{\circ}N$ and $125^{\circ}E/35^{\circ}N$, respectively. The output node of neural networks models consist of the daily precipitation data for Seoul station. For the performances of the neural networks models, they are composed of training and test performances, respectively. From this research, we evaluate the impact of GRNNM and MLP-NNM performances for the downscaling of the daily precipitation data. We should, therefore, construct the credible daily precipitation data for Seoul station using statistical downscaling method. The proposed methods can be applied to future climate prediction/projection using the various climate change scenarios such as GCMs and RCMs.

• Proceedings of the KIEE Conference
• /
• 1989.07a
• /
• pp.679-684
• /
• 1989

In this paper, we describe the PTP notion of robot manipulators by neural networks. The PTP motion requires the inverse kinematic redline and the joint trajectory generation algorithm. We use the multi-layered Perceptron neural networks and the Error Back Propagation(EBP) learning rule for inverse kinematic problems. Varying the number of hidden layers and the neurons of each hidden layer, we investigate the performance of the neural networks. Increasing the number of learning sweeps, we also discuss the performance of the neural networks. We propose a method for solving the inverse kinematic problems by adding the error compensation neural networks(ECNN). And, we implement the neural networks proposed by Grossberg et al. for automatic trajectory generation and discuss the problems in detail. Applying the neural networks to the current trajectory generation problems, we can refute the computation time for trajectory generation.

• Computers and Concrete
• /
• v.25 no.5
• /
• pp.401-409
• /
• 2020

Artificial neural networks are used as a useful tool in distinct fields of civil engineering these days. In order to control long-term quality of Self-Compacting Semi-Lightweight Concrete (SCSLC), the 90 days compressive strength is considered as a key issue in this paper. In fact, combined artificial neural networks are used to predict the compressive strength of SCSLC at 28 and 90 days. These networks are able to re-establish non-linear and complex relationships straightforwardly. In this study, two types of neural networks, including Radial Basis and Multilayer Perceptron, were used. Four groups of concrete mix designs also were made with two water to cement ratios (W/C) of 0.35 and 0.4, as well as 10% of cement weight was replaced with silica fume in half of the mixes, and different amounts of superplasticizer were used. With the help of rheology test and compressive strength results at 7 and 14 days as inputs, the neural networks were used to estimate the 28 and 90 days compressive strengths of above-mentioned mixes. It was necessary to add the 14 days compressive strength in the input layer to gain acceptable results for 90 days compressive strength. Then proper neural networks were prepared for each mix, following which four existing networks were combined, and the combinatorial neural network model properly predicted the compressive strength of different mix designs.

• Proceedings of the KIEE Conference
• /
• 2000.07d
• /
• pp.2949-2952
• /
• 2000

A new neural networks and learning algorithm are proposed in order to measure nonlinear heights of complexed road environments in realtime without pre-information. This new neural networks is Error Self Recurrent Neural Networks(ESRN), The structure of it is similar to recurrent neural networks: a delayed output as the input and a delayed error between the output of plant and neural networks as a bias input. In addition, we compute the desired value of hidden layer by an optimal method instead of transfering desired values by back-propagation and each weights are updated by RLS(Recursive Least Square). Consequently. this neural networks are not sensitive to initial weights and a learning rate, and have a faster convergence rate than conventional neural networks. We can estimate nonlinear models in realtime by ESRN and learning algorithm and control nonlinear models. To show the performance of this one. we control 7 degree of freedom full car model with several control method. From this simulation. this estimation and controller were proved to be effective to the measurements of nonlinear road environment systems.

• Journal of the Semiconductor & Display Technology
• /
• v.23 no.1
• /
• pp.1-6
• /
• 2024

Recently, artificial neural networks have been playing a crucial role and advancing across various fields. Artificial neural networks are typically categorized into feedforward neural networks and recurrent neural networks. However, feedforward neural networks are primarily used for processing static spatial patterns such as image recognition and object detection. They are not suitable for handling temporal signals. Recurrent neural networks, on the other hand, face the challenges of complex training procedures and requiring significant computational power. In this paper, we propose memristors suitable for an advanced form of recurrent neural networks called reservoir computing systems, utilizing a mask processor. Using the characteristic equations of Ti/TiOx/TaOy/Pt, Pt/TiOx/Pt, and Ag/ZnO-NW/Pt memristors, we generated current-voltage curves to verify their memristive behavior through the confirmation of hysteresis. Subsequently, we trained and inferred reservoir computing systems using these memristors with the NIST TI-46 database. Among these systems, the accuracy of the reservoir computing system based on Ti/TiOx/TaOy/Pt memristors reached 99%, confirming the Ti/TiOx/TaOy/Pt memristor structure's suitability for inferring speech recognition tasks.

• International journal of advanced smart convergence
• /
• v.9 no.2
• /
• pp.173-178
• /
• 2020

A common problem with neural network learning is that it is too suitable for the specificity of learning. In this paper, various methods were compared to avoid overfitting: regularization, drop-out, different numbers of data and different types of neural networks. Comparative studies of the above-mentioned methods have been provided to evaluate the test accuracy. I found that the more data using method is better than the regularization and dropout methods. Moreover, we know that deep convolutional neural networks outperform multi-layer neural networks and simple convolution neural networks.

• ETRI Journal
• /
• v.39 no.6
• /
• pp.832-840
• /
• 2017

Recently, deep recurrent neural networks have achieved great success in various machine learning tasks, and have also been applied for sound event detection. The detection of temporally overlapping sound events in realistic environments is much more challenging than in monophonic detection problems. In this paper, we present an approach to improve the accuracy of polyphonic sound event detection in multichannel audio based on gated recurrent neural networks in combination with auditory spectral features. In the proposed method, human hearing perception-based spatial and spectral-domain noise-reduced harmonic features are extracted from multichannel audio and used as high-resolution spectral inputs to train gated recurrent neural networks. This provides a fast and stable convergence rate compared to long short-term memory recurrent neural networks. Our evaluation reveals that the proposed method outperforms the conventional approaches.

• Proceedings of the IEEK Conference
• /
• 2000.07b
• /
• pp.1053-1056
• /
• 2000

In general, neural networks based modeling involves trying multiple networks with different architectures and/or training parameters in order to achieve the best accuracy. Only the single best-trained neural network is chosen, while the rest are discarded. However, using only the single best network may never give the best solution in every situation. Many researchers, therefore, propose methods to improve the accuracy of neural networks based modeling. In this paper, the idea of the logical combinations of neural networks is proposed and discussed in detail. The logical combination is constructed by combining the corresponding outputs of the neural networks with the logical “And” node. The experimental results based on simulated data show that the modeling accuracy is significantly improved when compared to using only the single best-trained neural network.

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 1998.06a
• /
• pp.410-412
• /
• 1998

Subsymbolic Knowledge processing is said to be changed states of networks constructed from small elements. subsymbolic systems also make it possible to use connectionist models for knowledge processing. Connectionist realization such modulus are modulus linked together for solving a given problem. We study using neural networks as distinct actions. The output vectors produced by the neural networks are consider as a new facts. These new facts are then processed to activate another networks or used in the current production rule, The production rule is applying knowledge stored in the knowledge base to make inference. After neural networks knowledge base is constructed and trained. We present a running sample of incorporating neural network knowledge base. We implement using rochester connectionist simulator. We suggest that incorporating neural network knowledge base. Therefore incorporated neural network knowledge base ensures a cleaner solution which results in better perfor s.