• Title/Summary/Keyword: Information entropy: Optimization-structure

Search Result 5, Processing Time 0.018 seconds

A Method for Optimizing the Structure of Neural Networks Based on Information Entropy

  • Yuan Hongchun;Xiong Fanlnu;Kei, Bai-Shi
    • Proceedings of the Korea Inteligent Information System Society Conference
    • /
    • 2001.01a
    • /
    • pp.30-33
    • /
    • 2001
  • The number of hidden neurons of the feed-forward neural networks is generally decided on the basis of experience. The method usually results in the lack or redundancy of hidden neurons, and causes the shortage of capacity for storing information of learning overmuch. This research proposes a new method for optimizing the number of hidden neurons bases on information entropy, Firstly, an initial neural network with enough hidden neurons should be trained by a set of training samples. Second, the activation values of hidden neurons should be calculated by inputting the training samples that can be identified correctly by the trained neural network. Third, all kinds of partitions should be tried and its information gain should be calculated, and then a decision-tree correctly dividing the whole sample space can be constructed. Finally, the important and related hidden neurons that are included in the tree can be found by searching the whole tree, and other redundant hidden neurons can be deleted. Thus, the number of hidden neurons can be decided. In the case of building a neural network with the best number of hidden units for tea quality evaluation, the proposed method is applied. And the result shows that the method is effective

  • PDF

Information entropy based algorithm of sensor placement optimization for structural damage detection

  • Ye, S.Q.;Ni, Y.Q.
    • Smart Structures and Systems
    • /
    • v.10 no.4_5
    • /
    • pp.443-458
    • /
    • 2012
  • The structural health monitoring (SHM) benchmark study on optimal sensor placement problem for the instrumented Canton Tower has been launched. It follows the success of the modal identification and model updating for the Canton Tower in the previous benchmark study, and focuses on the optimal placement of vibration sensors (accelerometers) in the interest of bettering the SHM system. In this paper, the sensor placement problem for the Canton Tower and the benchmark model for this study are first detailed. Then an information entropy based sensor placement method with the purpose of damage detection is proposed and applied to the benchmark problem. The procedure that will be implemented for structural damage detection using the data obtained from the optimal sensor placement strategy is introduced and the information on structural damage is specified. The information entropy based method is applied to measure the uncertainties throughout the damage detection process with the use of the obtained data. Accordingly, a multi-objective optimal problem in terms of sensor placement is formulated. The optimal solution is determined as the one that provides equally most informative data for all objectives, and thus the data obtained is most informative for structural damage detection. To validate the effectiveness of the optimally determined sensor placement, damage detection is performed on different damage scenarios of the benchmark model using the noise-free and noise-corrupted measured information, respectively. The results show that in comparison with the existing in-service sensor deployment on the structure, the optimally determined one is capable of further enhancing the capability of damage detection.

Community Detection using Closeness Similarity based on Common Neighbor Node Clustering Entropy

  • Jiang, Wanchang;Zhang, Xiaoxi;Zhu, Weihua
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.8
    • /
    • pp.2587-2605
    • /
    • 2022
  • In order to efficiently detect community structure in complex networks, community detection algorithms can be designed from the perspective of node similarity. However, the appropriate parameters should be chosen to achieve community division, furthermore, these existing algorithms based on the similarity of common neighbors have low discrimination between node pairs. To solve the above problems, a noval community detection algorithm using closeness similarity based on common neighbor node clustering entropy is proposed, shorted as CSCDA. Firstly, to improve detection accuracy, common neighbors and clustering coefficient are combined in the form of entropy, then a new closeness similarity measure is proposed. Through the designed similarity measure, the closeness similar node set of each node can be further accurately identified. Secondly, to reduce the randomness of the community detection result, based on the closeness similar node set, the node leadership is used to determine the most closeness similar first-order neighbor node for merging to create the initial communities. Thirdly, for the difficult problem of parameter selection in existing algorithms, the merging of two levels is used to iteratively detect the final communities with the idea of modularity optimization. Finally, experiments show that the normalized mutual information values are increased by an average of 8.06% and 5.94% on two scales of synthetic networks and real-world networks with real communities, and modularity is increased by an average of 0.80% on the real-world networks without real communities.

Optimal Buffer Allocation in Multi-Product Repairable Production Lines Based on Multi-State Reliability and Structural Complexity

  • Duan, Jianguo;Xie, Nan;Li, Lianhui
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.14 no.4
    • /
    • pp.1579-1602
    • /
    • 2020
  • In the design of production system, buffer capacity allocation is a major step. Through polymorphism analysis of production capacity and production capability, this paper investigates a buffer allocation optimization problem aiming at the multi-stage production line including unreliable machines, which is concerned with maximizing the system theoretical production rate and minimizing the system state entropy for a certain amount of buffers simultaneously. Stochastic process analysis is employed to establish Markov models for repairable modular machines. Considering the complex structure, an improved vector UGF (Universal Generating Function) technique and composition operators are introduced to construct the system model. Then the measures to assess the system's multi-state reliability and structural complexity are given. Based on system theoretical production rate and system state entropy, mathematical model for buffer capacity optimization is built and optimized by a specific genetic algorithm. The feasibility and effectiveness of the proposed method is verified by an application of an engine head production line.

Anomaly Detection for User Action with Generative Adversarial Networks (적대적 생성 모델을 활용한 사용자 행위 이상 탐지 방법)

  • Choi, Nam woong;Kim, Wooju
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.3
    • /
    • pp.43-62
    • /
    • 2019
  • At one time, the anomaly detection sector dominated the method of determining whether there was an abnormality based on the statistics derived from specific data. This methodology was possible because the dimension of the data was simple in the past, so the classical statistical method could work effectively. However, as the characteristics of data have changed complexly in the era of big data, it has become more difficult to accurately analyze and predict the data that occurs throughout the industry in the conventional way. Therefore, SVM and Decision Tree based supervised learning algorithms were used. However, there is peculiarity that supervised learning based model can only accurately predict the test data, when the number of classes is equal to the number of normal classes and most of the data generated in the industry has unbalanced data class. Therefore, the predicted results are not always valid when supervised learning model is applied. In order to overcome these drawbacks, many studies now use the unsupervised learning-based model that is not influenced by class distribution, such as autoencoder or generative adversarial networks. In this paper, we propose a method to detect anomalies using generative adversarial networks. AnoGAN, introduced in the study of Thomas et al (2017), is a classification model that performs abnormal detection of medical images. It was composed of a Convolution Neural Net and was used in the field of detection. On the other hand, sequencing data abnormality detection using generative adversarial network is a lack of research papers compared to image data. Of course, in Li et al (2018), a study by Li et al (LSTM), a type of recurrent neural network, has proposed a model to classify the abnormities of numerical sequence data, but it has not been used for categorical sequence data, as well as feature matching method applied by salans et al.(2016). So it suggests that there are a number of studies to be tried on in the ideal classification of sequence data through a generative adversarial Network. In order to learn the sequence data, the structure of the generative adversarial networks is composed of LSTM, and the 2 stacked-LSTM of the generator is composed of 32-dim hidden unit layers and 64-dim hidden unit layers. The LSTM of the discriminator consists of 64-dim hidden unit layer were used. In the process of deriving abnormal scores from existing paper of Anomaly Detection for Sequence data, entropy values of probability of actual data are used in the process of deriving abnormal scores. but in this paper, as mentioned earlier, abnormal scores have been derived by using feature matching techniques. In addition, the process of optimizing latent variables was designed with LSTM to improve model performance. The modified form of generative adversarial model was more accurate in all experiments than the autoencoder in terms of precision and was approximately 7% higher in accuracy. In terms of Robustness, Generative adversarial networks also performed better than autoencoder. Because generative adversarial networks can learn data distribution from real categorical sequence data, Unaffected by a single normal data. But autoencoder is not. Result of Robustness test showed that he accuracy of the autocoder was 92%, the accuracy of the hostile neural network was 96%, and in terms of sensitivity, the autocoder was 40% and the hostile neural network was 51%. In this paper, experiments have also been conducted to show how much performance changes due to differences in the optimization structure of potential variables. As a result, the level of 1% was improved in terms of sensitivity. These results suggest that it presented a new perspective on optimizing latent variable that were relatively insignificant.