• Title/Summary/Keyword: Recurrent Neural Network

Search Result 550, Processing Time 0.024 seconds

Recurrent Neural Network Adaptive Equalizers Based on Data Communication

  • Jiang, Hongrui;Kwak, Kyung-Sup
    • Journal of Communications and Networks
    • /
    • v.5 no.1
    • /
    • pp.7-18
    • /
    • 2003
  • In this paper, a decision feedback recurrent neural network equalizer and a modified real time recurrent learning algorithm are proposed, and an adaptive adjusting of the learning step is also brought forward. Then, a complex case is considered. A decision feedback complex recurrent neural network equalizer and a modified complex real time recurrent learning algorithm are proposed. Moreover, weights of decision feedback recurrent neural network equalizer under burst-interference conditions are analyzed, and two anti-burst-interference algorithms to prevent equalizer from out of working are presented, which are applied to both real and complex cases. The performance of the recurrent neural network equalizer is analyzed based on numerical results.

A New Recurrent Neural Network Architecture for Pattern Recognition and Its Convergence Results

  • Lee, Seong-Whan;Kim, Young-Joon;Song, Hee-Heon
    • Journal of Electrical Engineering and information Science
    • /
    • v.1 no.1
    • /
    • pp.108-117
    • /
    • 1996
  • In this paper, we propose a new type of recurrent neural network architecture in which each output unit is connected with itself and fully-connected with other output units and all hidden units. The proposed recurrent network differs from Jordan's and Elman's recurrent networks in view of functions and architectures because it was originally extended from the multilayer feedforward neural network for improving the discrimination and generalization power. We also prove the convergence property of learning algorithm of the proposed recurrent neural network and analyze the performance of the proposed recurrent neural network by performing recognition experiments with the totally unconstrained handwritten numeral database of Concordia University of Canada. Experimental results confirmed that the proposed recurrent neural network improves the discrimination and generalization power in recognizing spatial patterns.

  • PDF

Development of Basic Practice Cases for Recurrent Neural Networks (순환신경망 기초 실습 사례 개발)

  • Kyeong Hur
    • Journal of Practical Engineering Education
    • /
    • v.14 no.3
    • /
    • pp.491-498
    • /
    • 2022
  • In this paper, as a liberal arts course for non-major students, a case study of recurrent neural network SW practice, which is essential for designing a basic recurrent neural network subject curriculum, was developed. The developed SW practice case focused on understanding the operation principle of the recurrent neural network, and used a spreadsheet to check the entire visualized operation process. The developed recurrent neural network practice case consisted of creating supervised text completion training data, implementing the input layer, hidden layer, state layer (context node), and output layer in sequence, and testing the performance of the recurrent neural network on text data. The recurrent neural network practice case developed in this paper automatically completes words with various numbers of characters. Using the proposed recurrent neural network practice case, it is possible to create an artificial intelligence SW practice case that automatically completes by expanding the maximum number of characters constituting Korean or English words in various ways. Therefore, it can be said that the utilization of this case of basic practice of recurrent neural network is high.

Image Caption Generation using Recurrent Neural Network (Recurrent Neural Network를 이용한 이미지 캡션 생성)

  • Lee, Changki
    • Journal of KIISE
    • /
    • v.43 no.8
    • /
    • pp.878-882
    • /
    • 2016
  • Automatic generation of captions for an image is a very difficult task, due to the necessity of computer vision and natural language processing technologies. However, this task has many important applications, such as early childhood education, image retrieval, and navigation for blind. In this paper, we describe a Recurrent Neural Network (RNN) model for generating image captions, which takes image features extracted from a Convolutional Neural Network (CNN). We demonstrate that our models produce state of the art results in image caption generation experiments on the Flickr 8K, Flickr 30K, and MS COCO datasets.

A Study on the Bayesian Recurrent Neural Network for Time Series Prediction (시계열 자료의 예측을 위한 베이지안 순환 신경망에 관한 연구)

  • Hong Chan-Young;Park Jung-Hoon;Yoon Tae-Sung;Park Jin-Bae
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.10 no.12
    • /
    • pp.1295-1304
    • /
    • 2004
  • In this paper, the Bayesian recurrent neural network is proposed to predict time series data. A neural network predictor requests proper learning strategy to adjust the network weights, and one needs to prepare for non-linear and non-stationary evolution of network weights. The Bayesian neural network in this paper estimates not the single set of weights but the probability distributions of weights. In other words, the weights vector is set as a state vector of state space method, and its probability distributions are estimated in accordance with the particle filtering process. This approach makes it possible to obtain more exact estimation of the weights. In the aspect of network architecture, it is known that the recurrent feedback structure is superior to the feedforward structure for the problem of time series prediction. Therefore, the recurrent neural network with Bayesian inference, what we call Bayesian recurrent neural network (BRNN), is expected to show higher performance than the normal neural network. To verify the proposed method, the time series data are numerically generated and various kinds of neural network predictor are applied on it in order to be compared. As a result, feedback structure and Bayesian learning are better than feedforward structure and backpropagation learning, respectively. Consequently, it is verified that the Bayesian reccurent neural network shows better a prediction result than the common Bayesian neural network.

Bayesian Neural Network with Recurrent Architecture for Time Series Prediction

  • Hong, Chan-Young;Park, Jung-Hun;Yoon, Tae-Sung;Park, Jin-Bae
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2004.08a
    • /
    • pp.631-634
    • /
    • 2004
  • In this paper, the Bayesian recurrent neural network (BRNN) is proposed to predict time series data. Among the various traditional prediction methodologies, a neural network method is considered to be more effective in case of non-linear and non-stationary time series data. A neural network predictor requests proper learning strategy to adjust the network weights, and one need to prepare for non-linear and non-stationary evolution of network weights. The Bayesian neural network in this paper estimates not the single set of weights but the probability distributions of weights. In other words, we sets the weight vector as a state vector of state space method, and estimates its probability distributions in accordance with the Bayesian inference. This approach makes it possible to obtain more exact estimation of the weights. Moreover, in the aspect of network architecture, it is known that the recurrent feedback structure is superior to the feedforward structure for the problem of time series prediction. Therefore, the recurrent network with Bayesian inference, what we call BRNN, is expected to show higher performance than the normal neural network. To verify the performance of the proposed method, the time series data are numerically generated and a neural network predictor is applied on it. As a result, BRNN is proved to show better prediction result than common feedforward Bayesian neural network.

  • PDF

Recurrent Neural Network with Backpropagation Through Time Learning Algorithm for Arabic Phoneme Recognition

  • Ismail, Saliza;Ahmad, Abdul Manan
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2004.08a
    • /
    • pp.1033-1036
    • /
    • 2004
  • The study on speech recognition and understanding has been done for many years. In this paper, we propose a new type of recurrent neural network architecture for speech recognition, in which each output unit is connected to itself and is also fully connected to other output units and all hidden units [1]. Besides that, we also proposed the new architecture and the learning algorithm of recurrent neural network such as Backpropagation Through Time (BPTT, which well-suited. The aim of the study was to observe the difference of Arabic's alphabet like "alif" until "ya". The purpose of this research is to upgrade the people's knowledge and understanding on Arabic's alphabet or word by using Recurrent Neural Network (RNN) and Backpropagation Through Time (BPTT) learning algorithm. 4 speakers (a mixture of male and female) are trained in quiet environment. Neural network is well-known as a technique that has the ability to classified nonlinear problem. Today, lots of researches have been done in applying Neural Network towards the solution of speech recognition [2] such as Arabic. The Arabic language offers a number of challenges for speech recognition [3]. Even through positive results have been obtained from the continuous study, research on minimizing the error rate is still gaining lots attention. This research utilizes Recurrent Neural Network, one of Neural Network technique to observe the difference of alphabet "alif" until "ya".

  • PDF

Network Packet Classification Using Convolution Neural Network and Recurrent Neural Network (Convolution Neural Network와 Recurrent Neural Network를 활용한 네트워크 패킷 분류)

  • Lim, Hyun-Kyo;Kim, Ju-Bong;Han, Youn-Hee
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2018.05a
    • /
    • pp.16-18
    • /
    • 2018
  • 최근 네트워크 상에 새롭고 다양한 어플리케이션들이 생겨나면서 이에 따른 적절한 어플리케이션별 서비스 제공을 위한 패킷 분류 방법이 요구되고 있다. 이로 인하여 딥 러닝 기술이 발전 하면서 이를 이용한 네트워크 트래픽 분류 방법들이 제안되고 있다. 따라서, 본 논문에서는 딥 러닝 기술 중 Convolution Neural Network 와 Recurrent Neural Network 를 동시에 활용한 네트워크 패킷 분류 방법을 제안한다.

Identification of suspension systems using error self recurrent neural network and development of sliding mode controller (오차 자기 순환 신경회로망을 이용한 현가시스템 인식과 슬라이딩 모드 제어기 개발)

  • 송광현;이창구;김성중
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 1997.10a
    • /
    • pp.625-628
    • /
    • 1997
  • In this paper the new neural network and sliding mode suspension controller is proposed. That neural network is error self-recurrent neural network. For fast on-line learning, this paper use recursive least squares method. A new neural networks converges considerably faster than the backpropagation algorithm and has advantages of being less affected by the poor initial weights and learning rate. The controller for suspension systems is designed according to sliding mode technique based on new proposed neural network.

  • PDF

Equalization of Time-Varying Channels using a Recurrent Neural Network Trained with Kalman Filters (칼만필터로 훈련되는 순환신경망을 이용한 시변채널 등화)

  • 최종수;권오신
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.9 no.11
    • /
    • pp.917-924
    • /
    • 2003
  • Recurrent neural networks have been successfully applied to communications channel equalization. Major disadvantages of gradient-based learning algorithms commonly employed to train recurrent neural networks are slow convergence rates and long training sequences required for satisfactory performance. In a high-speed communications system, fast convergence speed and short training symbols are essential. We propose decision feedback equalizers using a recurrent neural network trained with Kalman filtering algorithms. The main features of the proposed recurrent neural equalizers, utilizing extended Kalman filter (EKF) and unscented Kalman filter (UKF), are fast convergence rates and good performance using relatively short training symbols. Experimental results for two time-varying channels are presented to evaluate the performance of the proposed approaches over a conventional recurrent neural equalizer.