• Title/Summary/Keyword: hidden unit number

Search Result 20, Processing Time 0.027 seconds

Parity Discrimination by Perceptron Neural Network (퍼셉트론형 신경회로망에 의한 패리티판별)

  • Choi, Jae-Seung
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.14 no.3
    • /
    • pp.565-571
    • /
    • 2010
  • This paper proposes a parity discrimination algorithm which discriminates N bit parity using a perceptron neural network and back propagation algorithm. This algorithm decides minimum hidden unit numbers when discriminates N bit parity. Therefore, this paper implements parity discrimination experiments for N bit by changing hidden unit numbers of the proposed perceptron neural network. Experiments confirm that the proposed algorithm is possible to discriminates N bit parity.

Isolated Word Recognition Using Allophone Unit Hidden Markov Model (변이음 HMM을 이용한 고립단어 인식)

  • Lee, Gang-Sung;Kim, Soon-Hyob
    • The Journal of the Acoustical Society of Korea
    • /
    • v.10 no.2
    • /
    • pp.29-35
    • /
    • 1991
  • In this paper, we discuss the method of recognizing allophone unit isolated words using hidden Markov model(HMM). Frist we constructed allophone lexicon by extracting allophones from training data and by training allophone HMMs. And then to recognize isolated words using allophone HMMs, it is necessary to construct word dictionary which contains information of allophone sequence and inter-allophone transition probability. Allophone sequences are represented by allophone HMMs. To see the effects of inter-allophone transition probability and to determine optimal probabilities, we performend some experiments. And we showed that small number of traing data and simple train procedure is needed to train word HMMs of allophone sequences and that not less performance than word unit HMM is obtained.

  • PDF

Bayesian Analysis for Neural Network Models

  • Chung, Younshik;Jung, Jinhyouk;Kim, Chansoo
    • Communications for Statistical Applications and Methods
    • /
    • v.9 no.1
    • /
    • pp.155-166
    • /
    • 2002
  • Neural networks have been studied as a popular tool for classification and they are very flexible. Also, they are used for many applications of pattern classification and pattern recognition. This paper focuses on Bayesian approach to feed-forward neural networks with single hidden layer of units with logistic activation. In this model, we are interested in deciding the number of nodes of neural network model with p input units, one hidden layer with m hidden nodes and one output unit in Bayesian setup for fixed m. Here, we use the latent variable into the prior of the coefficient regression, and we introduce the 'sequential step' which is based on the idea of the data augmentation by Tanner and Wong(1787). The MCMC method(Gibbs sampler and Metropolish algorithm) can be used to overcome the complicated Bayesian computation. Finally, a proposed method is applied to a simulated data.

Deep Dependence in Deep Learning models of Streamflow and Climate Indices

  • Lee, Taesam;Ouarda, Taha;Kim, Jongsuk;Seong, Kiyoung
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2021.06a
    • /
    • pp.97-97
    • /
    • 2021
  • Hydrometeorological variables contain highly complex system for temporal revolution and it is quite challenging to illustrate the system with a temporal linear and nonlinear models. In recent years, deep learning algorithms have been developed and a number of studies has focused to model the complex hydrometeorological system with deep learning models. In the current study, we investigated the temporal structure inside deep learning models for the hydrometeorological variables such as streamflow and climate indices. The results present a quite striking such that each hidden unit of the deep learning model presents different dependence structure and when the number of hidden units meet a proper boundary, it reaches the best model performance. This indicates that the deep dependence structure of deep learning models can be used to model selection or investigating whether the constructed model setup present efficient or not.

  • PDF

N bit Parity Discrimination using Perceptron Neural Network (신경회로망을 사용한 N 비트 패리티 판별)

  • Choi, Jae-seung
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2009.10a
    • /
    • pp.149-152
    • /
    • 2009
  • 본 논문에서는 오차역전파 알고리즘을 사용한 3층 구조의 퍼셉트론형 신경회로망으로 네트워크의 학습을 실시하여, N비트의 패리티판별에 필요한 최소의 중간유닛수의 해석에 관한 연구이다. 따라서 본 논문은 제안한 퍼셉트론형 신경회로망의 중간 유닛의 수를 변화시켜 N 비트의 패리티 판별 실험을 실시하였다. 본 시스템은 패리티 판별의 실험을 통하여 N 비트 패리티 판별이 가능하다는 것을 실험으로 확인한다.

  • PDF

Comparison of Artificial Neural Network Model Capability for Runoff Estimation about Activation Functions (활성화 함수에 따른 유출량 산정 인공신경망 모형의 성능 비교)

  • Kim, Maga;Choi, Jin-Yong;Bang, Jehong;Yoon, Pureun;Kim, Kwihoon
    • Journal of The Korean Society of Agricultural Engineers
    • /
    • v.63 no.1
    • /
    • pp.103-116
    • /
    • 2021
  • Analysis of runoff is substantial for effective water management in the watershed. Runoff occurs by reaction of a watershed to the rainfall and has non-linearity and uncertainty due to the complex relation of weather and watershed factors. ANN (Artificial Neural Network), which learns from the data, is one of the machine learning technique known as a proper model to interpret non-linear data. The performance of ANN is affected by the ANN's structure, the number of hidden layer nodes, learning rate, and activation function. Especially, the activation function has a role to deliver the information entered and decides the way of making output. Therefore, It is important to apply appropriate activation functions according to the problem to solve. In this paper, ANN models were constructed to estimate runoff with different activation functions and each model was compared and evaluated. Sigmoid, Hyperbolic tangent, ReLU (Rectified Linear Unit), ELU (Exponential Linear Unit) functions were applied to the hidden layer, and Identity, ReLU, Softplus functions applied to the output layer. The statistical parameters including coefficient of determination, NSE (Nash and Sutcliffe Efficiency), NSEln (modified NSE), and PBIAS (Percent BIAS) were utilized to evaluate the ANN models. From the result, applications of Hyperbolic tangent function and ELU function to the hidden layer and Identity function to the output layer show competent performance rather than other functions which demonstrated the function selection in the ANN structure can affect the performance of ANN.

Connected Korean Digit Speech Recognition Using Vowel String and Number of Syllables (음절수와 모음 열을 이용한 한국어 연결 숫자 음성인식)

  • Youn, Jeh-Seon;Hong, Kwang-Seok
    • The KIPS Transactions:PartA
    • /
    • v.10A no.1
    • /
    • pp.1-6
    • /
    • 2003
  • In this paper, we present a new Korean connected digit recognition based on vowel string and number of syllables. There are two steps to reduce digit candidates. The first one is to determine the number and interval of digit. Once the number and interval of digit are determined, the second is to recognize the vowel string in the digit string. The digit candidates according to vowel string are recognized based on CV (consonant vowel), VCCV and VC unit HMM. The proposed method can cope effectively with the coarticulation effects and recognize the connected digit speech very well.

Control of Nonlinear System by Multiplication and Combining Layer on Dynamic Neural Networks (동적 신경망의 층의 분열과 합성에 의한 비선형 시스템 제어)

  • Park, Seong-Wook;Lee, Jae-Kwan;Seo, Bo-Hyeok
    • The Transactions of the Korean Institute of Electrical Engineers A
    • /
    • v.48 no.4
    • /
    • pp.419-427
    • /
    • 1999
  • We propose an algorithm for obtaining the optimal node number of hidden units in dynamic neural networks. The dynamic nerual networks comprise of dynamic neural units and neural processor consisting of two dynamic neural units; one functioning as an excitatory neuron and the other as an inhibitory neuron. Starting out with basic network structure to solve the problem of control, we find optimal neural structure by multiplication and combining dynamic neural unit. Numerical examples are presented for nonlinear systems. Those case studies showed that the proposed is useful is practical sense.

  • PDF

Anomaly Detection for User Action with Generative Adversarial Networks (적대적 생성 모델을 활용한 사용자 행위 이상 탐지 방법)

  • Choi, Nam woong;Kim, Wooju
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.3
    • /
    • pp.43-62
    • /
    • 2019
  • At one time, the anomaly detection sector dominated the method of determining whether there was an abnormality based on the statistics derived from specific data. This methodology was possible because the dimension of the data was simple in the past, so the classical statistical method could work effectively. However, as the characteristics of data have changed complexly in the era of big data, it has become more difficult to accurately analyze and predict the data that occurs throughout the industry in the conventional way. Therefore, SVM and Decision Tree based supervised learning algorithms were used. However, there is peculiarity that supervised learning based model can only accurately predict the test data, when the number of classes is equal to the number of normal classes and most of the data generated in the industry has unbalanced data class. Therefore, the predicted results are not always valid when supervised learning model is applied. In order to overcome these drawbacks, many studies now use the unsupervised learning-based model that is not influenced by class distribution, such as autoencoder or generative adversarial networks. In this paper, we propose a method to detect anomalies using generative adversarial networks. AnoGAN, introduced in the study of Thomas et al (2017), is a classification model that performs abnormal detection of medical images. It was composed of a Convolution Neural Net and was used in the field of detection. On the other hand, sequencing data abnormality detection using generative adversarial network is a lack of research papers compared to image data. Of course, in Li et al (2018), a study by Li et al (LSTM), a type of recurrent neural network, has proposed a model to classify the abnormities of numerical sequence data, but it has not been used for categorical sequence data, as well as feature matching method applied by salans et al.(2016). So it suggests that there are a number of studies to be tried on in the ideal classification of sequence data through a generative adversarial Network. In order to learn the sequence data, the structure of the generative adversarial networks is composed of LSTM, and the 2 stacked-LSTM of the generator is composed of 32-dim hidden unit layers and 64-dim hidden unit layers. The LSTM of the discriminator consists of 64-dim hidden unit layer were used. In the process of deriving abnormal scores from existing paper of Anomaly Detection for Sequence data, entropy values of probability of actual data are used in the process of deriving abnormal scores. but in this paper, as mentioned earlier, abnormal scores have been derived by using feature matching techniques. In addition, the process of optimizing latent variables was designed with LSTM to improve model performance. The modified form of generative adversarial model was more accurate in all experiments than the autoencoder in terms of precision and was approximately 7% higher in accuracy. In terms of Robustness, Generative adversarial networks also performed better than autoencoder. Because generative adversarial networks can learn data distribution from real categorical sequence data, Unaffected by a single normal data. But autoencoder is not. Result of Robustness test showed that he accuracy of the autocoder was 92%, the accuracy of the hostile neural network was 96%, and in terms of sensitivity, the autocoder was 40% and the hostile neural network was 51%. In this paper, experiments have also been conducted to show how much performance changes due to differences in the optimization structure of potential variables. As a result, the level of 1% was improved in terms of sensitivity. These results suggest that it presented a new perspective on optimizing latent variable that were relatively insignificant.

A Structured and Multi-cellular Model of Starch Biosynthesis in Potato

  • Saithong, Treenut;Saraboon, Piyaporn;Meechai, Asawin;Cheevadhanarak, Supapon;Bhumiratana, Sakarindr
    • Proceedings of the Korean Society for Bioinformatics Conference
    • /
    • 2005.09a
    • /
    • pp.151-155
    • /
    • 2005
  • Recently, systems biology has been increasingly applied to gain insights into the complexity of living organisms. Many inaccessible biological information and hidden evidences fur example flux distribution of the metabolites are simply revealed by investigation of artificial cell behaviors. Most bio-models are models of single cell organisms that cannot handle the multi-cellular organisms like plants. Herein, a structured and multi-cellular model of potato was developed to comprehend the root starch biosynthesis. On the basis of simplest plant cell biology, a potato structured model on the platform of Berkley Madonna was divided into three parts: photosynthetic (leaf), non-photosynthetic (tuber) and transportation (phloem) cells. The model of starch biosynthesis begins with the fixation of CO$_2$ from atmosphere to the Calvin cycle. Passing through a series of reactions, triose phosphate from Calvin cycle is converted to sucrose which is transported to sink cells and is eventually formed the amylose and amylopectin (starch constituents). After validating the model with data from a number of literatures, the results show that the structured model is a good representative of the studied system. The result of triose phosphate (DHAP and GAP) elevation due to lessening the aldolase activity is an illustration of the validation. Furthermore, the representative model was used to gain more understanding of starch production process such as the effect of CO$_2$ uptake on qualitative and quantitative aspects of starch biosynthesis.

  • PDF