• Journal of the Korean Institute of Telematics and Electronics B
• /
• v.31B no.4
• /
• pp.145-154
• /
• 1994

This paper proposes an efficient method for improving the training performance of the neural network by using a hybrid of a stochastic approximation and a backpropagation algorithm. The proposed method improves the performance of the training by appliying a global optimization method which is a hybrid of a stochastic approximation and a backpropagation algorithm. The approximate initial point for a stochastic approximation and a backpropagation algorihtm. The approximate initial point for fast global optimization is estimated first by applying the stochastic approximation, and then the backpropagation algorithm, which is the fast gradient descent method, is applied for a high speed global optimization. And further speed-up of training is made possible by adjusting the training parameters of each of the output and the hidden layer adaptively to the standard deviation of the neuron output of each layer. The proposed method has been applied to the parity checking and the pattern classification, and the simulation results show that the performance of the proposed method is superior to that of the backpropagation, the Baba's MROM, and the Sun's method with randomized initial point settings. The results of adaptive adjusting of the training parameters show that the proposed method further improves the convergence speed about 20% in training.

• KIPS Transactions on Computer and Communication Systems
• /
• v.10 no.7
• /
• pp.191-198
• /
• 2021

Since the size of training dataset become large and the model is getting deeper to achieve high accuracy in deep learning, the deep neural network training requires a lot of computation and it takes too much time with a single node. Therefore, distributed deep learning is proposed to reduce the training time by distributing computation across multiple nodes. In this study, we propose hybrid allreduce strategy that considers the characteristics of each layer and communication and computational overlapping technique for synchronization of distributed deep learning. Since the convolution layer has fewer parameters than the fully-connected layer as well as it is located at the upper, only short overlapping time is allowed. Thus, butterfly allreduce is used to synchronize the convolution layer. On the other hand, fully-connecter layer is synchronized using ring all-reduce. The empirical experiment results on PyTorch with our proposed scheme shows that the proposed method reduced the training time by up to 33% compared to the baseline PyTorch.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2022.05a
• /
• pp.516-518
• /
• 2022

In this paper, we introduce a dependence of number of nodes of hidden-layer in fall detection system using Long Short-Term Memory that can detect falls. Its training is carried out using the parameter theta(θ), which indicates the angle formed by the x, y, and z-axis data for the direction of gravity using a 3-axis acceleration sensor. In its learning, validation is performed and divided into training data and test data in a ratio of 8:2, and training is performed by changing the number of nodes in the hidden layer to increase efficiency. When the number of nodes is 128, the best accuracy is shown with Accuracy = 99.82%, Specificity = 99.58%, and Sensitivity = 100%.

• Journal of Mechanical Science and Technology
• /
• v.20 no.12
• /
• pp.2079-2086
• /
• 2006

A fundamental study for developing a fault diagnosis system of a pump is performed by using neural network. Acoustic signals were obtained and converted to frequency domain for normal products and artificially deformed products. The neural network model used in this study was 3-layer type composed of input, hidden, and output layer. The normalized amplitudes at the multiples of real driving frequency were chosen as units of input layer. And the codes of pump malfunctions were selected as units of output layer. Various sets of teach signals made from original data by eliminating some random cases were used in the training. The average errors were approximately proportional to the number of untaught data. Neural network trained by acoustic signals can detect malfunction or diagnose fault of a given machine from the results.

• Journal of Digital Contents Society
• /
• v.19 no.6
• /
• pp.1197-1205
• /
• 2018

In this paper, we perform Single Image Super Resolution(SISR) for acquired images of EOTS or IRST from naval combat system. In order to conduct super resolution, we use Generative Adversarial Networks(GANs), which consists of a generative model to create a super-resolution image from the given low-resolution image and a discriminative model to determine whether the generated super-resolution image is qualified as a high-resolution image by adjusting various learning parameters. The learning parameters consist of a crop size of input image, the depth of sub-pixel layer, and the types of training images. Regarding evaluation method, we apply not only general image quality metrics, but feature descriptor methods. As a result, a larger crop size, a deeper sub-pixel layer, and high-resolution training images yield good performance.

• The KIPS Transactions:PartB
• /
• v.13B no.6 s.109
• /
• pp.615-624
• /
• 2006

The objective of this paper is to implement parallel multi-layer ANN(Artificial Neural Network) simulator based on the mobile agent system which is executed in parallel in the virtual parallel distributed computing environment. The Multi-Layer Neural Network is classified by training session, training data layer, node, md weight in the parallelization-level. In this study, We have developed and evaluated the simulator with which it is feasible to parallel the ANN in the training session and training data parallelization because these have relatively few network traffic. In this results, we have verified that the performance of parallelization is high about 3.3 times in the training session and training data. The great significance of this paper is that the performance of ANN's execution on virtual parallel computer is similar to that of ANN's execution on existing super-computer. Therefore, we think that the virtual parallel computer can be considerably helpful in developing the neural network because it decreases the training time which needs extra-time.

• Journal of the Korea Society of Computer and Information
• /
• v.9 no.4 s.32
• /
• pp.33-39
• /
• 2004

In this paper. we design the learning algorithm of LVQ which is used Forward Counter Propagation Networks to improve classification performance of LVQ networks. The weights of Forward Counter Propagation Networks which is between input layer and cluster layer can be learned to determine initial reference vectors by using SOM algorithm and to learn reference vectors by using LVQ algorithm. Finally. pattern vectors is classified into subclasses by neurons which is being in the cluster layer, and the weights of Forward Counter Propagation Networks which is between cluster layer and output layer is learned to classify the classified subclass, which is enclosed a class. Also. kr the number of classes is determined, the number of neurons which is being in the input layer, cluster layer and output layer can be determined. To prove the performance of the proposed learning algorithm. the simulation is performed by using training vectors and test vectors that ate Fisher's Iris data, and classification performance of the proposed learning method is compared with ones of the conventional LVQ, and it was a confirmation that the proposed learning method is more successful classification than the conventional classification.

• Journal of the Korean Society of Fisheries and Ocean Technology
• /
• v.33 no.2
• /
• pp.141-148
• /
• 1997

This paper is concerned with the vibration control of multi-layer structure for ultra-tall buildings and main tower of large bridge etc. We have modeled the multi-layer structure with the distributed mass system as the lumped mass system of two-degree-of-freedom structure and made experimental equipment. The LQ optimal control theory is applied to the design of the control system. The designed control system is simulated by computer. As a result, the LQ regulator showed good vibration control performance with impact excitation.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.5 no.4
• /
• pp.766-776
• /
• 2001

In the constructing of the multi layer neural network, the network topology is often chosen arbitrarily for different applications, and the optimum topology of the network is determined by the long processing of the trial and error. In this paper, we propose the automatic topology construction using the fuzzy rule that optimizes the neurons of hidden layer, and prune the weights connecting the hidden layer and the output layer during the training process. The simulation of pattern recognition, and the experiment of the mapping of the inverted pendulum showed the effectiveness of the proposed method.

• Structural Engineering and Mechanics
• /
• v.13 no.2
• /
• pp.187-209
• /
• 2002

The infrastructure system in the United States has been aging faster than the resource available to restore them. Therefore decision for allocating the resources is based in part on the condition of the structural system. This paper proposes to use neural network to predict the overall rating of the structural system because of the successful applications of neural network to other fields which require a "symptom-diagnostic" type relationship. The goal of this paper is to illustrate the potential of using neural network in civil engineering applications and, particularly, in bridge evaluations. Data collected by the Tennessee Department of Transportation were used as "test bed" for the study. Multi-layer feed forward networks were developed using the Levenberg-Marquardt training algorithm. All the neural networks consisted of at least one hidden layer of neurons. Hyperbolic tangent transfer functions were used in the first hidden layer and log-sigmoid transfer functions were used in the subsequent hidden and output layers. The best performing neural network consisted of three hidden layers. This network contained three neurons in the first hidden layer, two neurons in the second hidden layer and one neuron in the third hidden layer. The neural network performed well based on a target error of 10%. The results of this study indicate that the potential for using neural networks for the evaluation of infrastructure systems is very good.