• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.3
• /
• pp.837-852
• /
• 2021

Recurrent neural network (RNN) architectures have been used for language modeling (LM) tasks that require learning long-range word or character sequences. However, the RNN architecture is still suffered from unstable gradients on long-range sequences. To address the issue of long-range sequences, an attention mechanism has been used, showing state-of-the-art (SOTA) performance in all LM tasks. A differentiable neural computer (DNC) is a deep learning architecture using an attention mechanism. The DNC architecture is a neural network augmented with a content-addressable external memory. However, in the write operation, some information unrelated to the input word remains in memory. Moreover, DNCs have been found to perform poorly with low numbers of weight parameters. Therefore, we propose a robust memory deallocation method using a limited retention vector. The limited retention vector determines whether the network increases or decreases its usage of information in external memory according to a threshold. We experimentally evaluate the robustness of a DNC implementing the proposed approach according to the size of the controller and external memory on the enwik8 LM task. When we decreased the number of weight parameters by 32.47%, the proposed DNC showed a low bits-per-character (BPC) degradation of 4.30%, demonstrating the effectiveness of our approach in language modeling tasks.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.42 no.5 s.305
• /
• pp.165-172
• /
• 2005

In speech recognition under a noisy environment, it is necessary to construct a system which reduces the noise and enhances the speech. Then it is effective to imitate the human auditory system which has an excellent analytical spectrum mechanism for speech enhancement. Accordingly, this paper proposes an adaptive method using the auditory mechanism which is called lateral inhibition. This method first estimates the noise intensity by neural network, then adaptively adjusts both the coefficients of the lateral inhibition and the adjusting coefficient of amplitude component according to the noise intensity for each input frame. It is confirmed that the proposed method is effective for speech degraded by white noise, colored noise, and road noise based on the spectral distortion measurement.

• Journal of Institute of Control, Robotics and Systems
• /
• v.10 no.12
• /
• pp.1137-1147
• /
• 2004

In this paper, we proposed two types of adaptive control mechanism which is named HIA(Humoral Immune Algorithm) PID and CMIA(Cell-Mediated Immune Algorithm) controller based on biological immune system under engineering point of view. The HIA PID which has real time control scheme is focused on the humoral immunity and the latter which has the self-tuning mechanism is focused on the T-cell regulated immune response. To verify the performance of the proposed controller, some experiments for the control of AGV which is used for the port automation to carry container without human are performed. The experimental results for the control of steering and speed of an AGV system illustrate the effectiveness of the proposed control scheme. Moreover, in that results, proposed controllers have better performance than other conventional PID controller and intelligent control method which is the NN(neural network) PID controller.

• Journal of Biomedical Engineering Research
• /
• v.2 no.2
• /
• pp.127-140
• /
• 1981

This paper describes the mechanism of information processing in the nervous system through neuron pool model which is consisted of six single neural models. In the neuron pool model, summation characteristic of stimulus satisfies those of real nervous system and output impulse rate increases linearly to the input stimulus. Occlusion phenomena of the neuron pool model is approached to those of real nervous system and also if the threshold potential within sutlirninal fringe is increased, facilitation phenomena appreared. Therefore, the results of this study suggest that we can construct large neuron pool with many single neural models and verify the mechanism of information processing in the wide part of nervous system.

• Proceedings of the IEEK Conference
• /
• 1998.06a
• /
• pp.126-129
• /
• 1998

In order to control the flow of traffics in ATM networks and optimize the usage of network resources, an efficient control mechanism is necessary to cope with congestion and prevent the degradation of network performance caused by congestion. This paper proposes a new UPC(Usage Parameter Control) mechanism that varies the token generation rate and the buffer threshold of leaky bucket by using a Neural Network controller observing input buffers and token pools, thus achieving the improvement of performance. Simulation results show that the proposed adaptive algorithm uses of network resources efficiently and satisfies QoS for the various kinds of traffics.

• Physical Therapy Korea
• /
• v.2 no.2
• /
• pp.98-107
• /
• 1995

Physical therapists use assumptions about motor control in every aspect of their work in treating stroke patients. An understanding of the recovery process after stroke, some neural mechanism of recovery and therapeutic model is critical factor for physical therapist to evaluate and obtain a higher final stage of recovery. The purpose of this article was to review the recovery process after stroke, some neural mechanism of recovery, the role of rehabilitation in the process of recovery, therapeutic model and its limitation. This article will help understanding of recovery process. evaluation, and treatment of the stroke patients. Each therapeutic method consists of a different set of assumptions and they are not completely independent of one another. Therefore specializing in any techniques of physical therapy will not be enough to treat stroke, so we are in need of integrated approach and objective measurement instrument to adequately evaluate and treat stroke patients.

• Computers and Concrete
• /
• v.26 no.5
• /
• pp.411-420
• /
• 2020

Deep Convolution neural network (DCNN) has been widely used in the healthy maintenance of civil infrastructure. Using DCNN to improve crack detection performance has attracted many researchers' attention. In this paper, a light-weight spatial attention network module is proposed to strengthen the representation capability of ResNet and improve the crack detection performance. It utilizes attention mechanism to strengthen the interested objects in global receptive field of ResNet convolution layers. Global average spatial information over all channels are used to construct an attention scalar. The scalar is combined with adaptive weighted sigmoid function to activate the output of each channel's feature maps. Salient objects in feature maps are refined by the attention scalar. The proposed spatial attention module is stacked in ResNet50 to detect crack. Experiments results show that the proposed module can got significant performance improvement in crack detection.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2003.10a
• /
• pp.188-191
• /
• 2003

In this paper, we proposed data generalization and data cube mechanism for efficient data mining in distribute environment. We also proposed active Self Organization Map applying traditional Self Organization Map of Neural network for searching the most Informative data created from data cube after the generalization procedure and designed the system architecture for that.

• The Transactions of the Korea Information Processing Society
• /
• v.1 no.3
• /
• pp.311-318
• /
• 1994

In this paper, a new stroke extraction method of Chinese character base on the human optical field(the Receptive Field of Cell) is proposed. In processing the feature extraction of the chinese character, needed are more perfect extraction methods for separated informations and its data base. This method can be applied to processing neural cell using conventional feature extraction mechanism in the optical boundary of retina and cerebrum. With this method, its applicability and effectiveness were demonstrated extracting strokes from Chinese character.

• Journal of Information Processing Systems
• /
• v.17 no.2
• /
• pp.399-410
• /
• 2021

Aiming at the problem that complex external variables in natural scenes have a greater impact on facial expression recognition results, a facial expression recognition method based on two-stream convolutional neural network is proposed. The model introduces exponentially enhanced shared input weights before each level of convolution input, and uses soft attention mechanism modules on the space-time features of the combination of static and dynamic streams. This enables the network to autonomously find areas that are more relevant to the expression category and pay more attention to these areas. Through these means, the information of irrelevant interference areas is suppressed. In order to solve the problem of poor local robustness caused by lighting and expression changes, this paper also performs lighting preprocessing with the lighting preprocessing chain algorithm to eliminate most of the lighting effects. Experimental results on AFEW6.0 and Multi-PIE datasets show that the recognition rates of this method are 95.05% and 61.40%, respectively, which are better than other comparison methods.