• International Journal of Contents
• /
• v.10 no.1
• /
• pp.18-22
• /
• 2014

Hidden nodes have a key role in the information processing of feed-forward neural networks in which inputs are processed through a series of weighted sums and nonlinear activation functions. In order to understand the role of hidden nodes, we must analyze the effect of the nonlinear activation functions on the weighted sums to hidden nodes. In this paper, we focus on the effect of nonlinear functions in a viewpoint of information theory. Under the assumption that the nonlinear activation function can be approximated piece-wise linearly, we prove that the entropy of weighted sums to hidden nodes decreases after piece-wise linear functions. Therefore, we argue that the nonlinear activation function decreases the uncertainty among hidden nodes. Furthermore, the more the hidden nodes are saturated, the more the entropy of hidden nodes decreases. Based on this result, we can say that, after successful training of feed-forward neural networks, hidden nodes tend not to be in linear regions but to be in saturated regions of activation function with the effect of uncertainty reduction.

• Proceedings of the Korea Inteligent Information System Society Conference
• /
• 2001.01a
• /
• pp.30-33
• /
• 2001

The number of hidden neurons of the feed-forward neural networks is generally decided on the basis of experience. The method usually results in the lack or redundancy of hidden neurons, and causes the shortage of capacity for storing information of learning overmuch. This research proposes a new method for optimizing the number of hidden neurons bases on information entropy, Firstly, an initial neural network with enough hidden neurons should be trained by a set of training samples. Second, the activation values of hidden neurons should be calculated by inputting the training samples that can be identified correctly by the trained neural network. Third, all kinds of partitions should be tried and its information gain should be calculated, and then a decision-tree correctly dividing the whole sample space can be constructed. Finally, the important and related hidden neurons that are included in the tree can be found by searching the whole tree, and other redundant hidden neurons can be deleted. Thus, the number of hidden neurons can be decided. In the case of building a neural network with the best number of hidden units for tea quality evaluation, the proposed method is applied. And the result shows that the method is effective

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.24 no.9B
• /
• pp.1775-1784
• /
• 1999

This paper proposes a method of removing unnecessary hidden nodes by a new cost function that evaluates the variance and the mean of hidden node outputs during training. The proposed cost function makes necessary hidden nodes be activated and unnecessary hidden nodes be constants. We can remove the constant hidden nodes without performance degradation. Using the CEDAR handwritten digit recognition, we have shown that the proposed method can remove the number of hidden nodes up to 37.2%, with higher recognition rate and shorter learning time.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.10 no.7
• /
• pp.3150-3170
• /
• 2016

In Intelligent Transportation Systems (ITS), logistics distribution and mobile e-commerce, the real-time, accurate and reliable vehicle trajectory prediction has significant application value. Vehicle trajectory prediction can not only provide accurate location-based services, but also can monitor and predict traffic situation in advance, and then further recommend the optimal route for users. In this paper, firstly, we mine the double layers of hidden states of vehicle historical trajectories, and then determine the parameters of HMM (hidden Markov model) by historical data. Secondly, we adopt Viterbi algorithm to seek the double layers hidden states sequences corresponding to the just driven trajectory. Finally, we propose a new algorithm (DHMTP) for vehicle trajectory prediction based on the hidden Markov model of double layers hidden states, and predict the nearest neighbor unit of location information of the next k stages. The experimental results demonstrate that the prediction accuracy of the proposed algorithm is increased by 18.3% compared with TPMO algorithm and increased by 23.1% compared with Naive algorithm in aspect of predicting the next k phases' trajectories, especially when traffic flow is greater, such as this time from weekday morning to evening. Moreover, the time performance of DHMTP algorithm is also clearly improved compared with TPMO algorithm.

• Proceedings of the KSPS conference
• /
• 2005.04a
• /
• pp.87-90
• /
• 2005

Today's state-of the-art speech recognition systems typically use continuous distribution hidden Markov models with the mixtures of Gaussian distributions. To obtain higher recognition accuracy, the hidden Markov models typically require huge number of Gaussian distributions. Such speech recognition systems have problems that they require too much memory to run, and are too slow for large applications. Many approaches are proposed for the design of compact acoustic models. One of those models is subspace distribution clustering hidden Markov model. Subspace distribution clustering hidden Markov model can represent original full-space distributions as some combinations of a small number of subspace distribution codebooks. Therefore, how to make the codebook is an important issue in this approach. In this paper, we report some experimental results on various quantization methods to make more accurate models.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.23 no.12
• /
• pp.1742-1745
• /
• 2019

Hidden flame detection after allegedly extinguishing a fire cannot be emphasized enough. There are a few commercial hidden fire detection equipments which are imported, but the cost is relatively high. In this contribution, we propose a development of a low-cost, high-performance hidden flame detector using open-source hardware/software. We use Raspberry-pi based hardware board equipped with a TFT touch-screen LCD, a 3G modem, and an attachable battery device altogether integrated in a plastic case fabricated with a 3D printer. The proposed hidden flame detector shows the same performance of a commercial product FLIR E5 while consuming less than a half of the cost.

• Proceedings of the IEEK Conference
• /
• 2002.07a
• /
• pp.258-261
• /
• 2002

This paper addresses the problem of merging two feedforward neural networks into one network. Merging is accomplished at the level of hidden layer. A new network selects its hidden layer's units from the two networks to be merged We uses information theoretic criterion (quadratic mutual information) in the selection process. The hidden unit's output and the target patterns are considers as random variables and the mutual information between them is calculated. The mutual information between hidden units are also considered to prevent the statistically dependent units from being selected. Because mutual information is invariant under linear transformation of the variables, it shows the property of the robust estimation.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.18 no.2
• /
• pp.243-250
• /
• 2008

This paper proposes a system which extracts necessary information from call-for-paper (CFP) documents using a hidden Markov model (HMM). Even though a CFP does not follow a strict form, there is, in general, a relatively-fixed sequence of information within most CFPs. Therefore, a hiden Markov model is adopted to analyze CFPs which has an advantage of processing consecutive data. However, when CFPs are intuitively modeled with a hidden Markov model, a problem arises that the boundaries of the information are not recognized accurately. In order to solve this problem, this paper proposes a two-phrase hidden Markov model. In the first step, the P-HMM (Phrase hidden Markov model) which models a document with phrases recognizes CFP documents locally. Then, the D-HMM (Document hidden Markov model) grasps the overall structure and information flow of the document. The experiments over 400 CFP documents grathered on Web result in 0.49 of F-score. This performance implies 0.15 of F-measure improvement over the HMM which is intuitively modeled.

• Journal of the Optical Society of Korea
• /
• v.18 no.6
• /
• pp.706-713
• /
• 2014

An enhanced 360-degree integral-floating three-dimensional display system using a hexagonal lens array and a hidden point removal operator is proposed. Only the visible points of the chosen three-dimensional point cloud model are detected by the hidden point removal operator for each rotating step of the anamorphic optics system, and elemental image arrays are generated for the detected visible points from the corresponding viewpoint. Each elemental image of the elemental image array is generated by a hexagonal grid, due to being captured through a hexagonal lens array. The hidden point removal operator eliminates the overlap problem of points in front and behind, and the hexagonal lens array captures the elemental image arrays with more accurate approximation, so in the end the quality of the displayed image is improved. In an experiment, an anamorphic-optics-system-based 360-degree integral-floating display with improved image quality is demonstrated.

• Journal of Electrical Engineering and information Science
• /
• v.2 no.6
• /
• pp.7-13
• /
• 1997

Morphological Analysis of Korean has known to be a very complicated problem. Especially, the degree of part-of-speech(POS) ambiguity is much higher than English. Many researchers have tried to use a hidden Markov model(HMM) to solve the POS tagging problem and showed arround 95% correctness ratio. However, the lack of lexical information involves a hidden Markov model for POS tagging in lots of difficulties in improving the performance. To alleviate the burden, this paper proposes a method for combining multiword units, which are types of lexical information, into a hidden Markov model for POS tagging. This paper also proposes a method for extracting multiword units from POS tagged corpus. In this paper, a multiword unit is defined as a unit which consists of more than one word. We found that these multiword units are the major source of POS tagging errors. Our experiment shows that the error reduction rate of the proposed method is about 13%.