• Title/Summary/Keyword: hidden layer

Search Result 512, Processing Time 0.025 seconds

Neural Network Image Reconstruction for Magnetic Particle Imaging

  • Chae, Byung Gyu
    • ETRI Journal
    • /
    • v.39 no.6
    • /
    • pp.841-850
    • /
    • 2017
  • We investigate neural network image reconstruction for magnetic particle imaging. The network performance strongly depends on the convolution effects of the spectrum input data. The larger convolution effect appearing at a relatively smaller nanoparticle size obstructs the network training. The trained single-layer network reveals the weighting matrix consisting of a basis vector in the form of Chebyshev polynomials of the second kind. The weighting matrix corresponds to an inverse system matrix, where an incoherency of basis vectors due to low convolution effects, as well as a nonlinear activation function, plays a key role in retrieving the matrix elements. Test images are well reconstructed through trained networks having an inverse kernel matrix. We also confirm that a multi-layer network with one hidden layer improves the performance. Based on the results, a neural network architecture overcoming the low incoherence of the inverse kernel through the classification property is expected to become a better tool for image reconstruction.

Robust Multi-Layer Hierarchical Model for Digit Character Recognition

  • Yang, Jie;Sun, Yadong;Zhang, Liangjun;Zhang, Qingnian
    • Journal of Electrical Engineering and Technology
    • /
    • v.10 no.2
    • /
    • pp.699-707
    • /
    • 2015
  • Although digit character recognition has got a significant improvement in recent years, it is still challenging to achieve satisfied result if the data contains an amount of distracting factors. This paper proposes a novel digit character recognition approach using a multi-layer hierarchical model, Hybrid Restricted Boltzmann Machines (HRBMs), which allows the learning architecture to be robust to background distracting factors. The insight behind the proposed model is that useful high-level features appear more frequently than distracting factors during learning, thus the high-level features can be decompose into hybrid hierarchical structures by using only small label information. In order to extract robust and compact features, a stochastic 0-1 layer is employed, which enables the model's hidden nodes to independently capture the useful character features during training. Experiments on the variations of Mixed National Institute of Standards and Technology (MNIST) dataset show that improvements of the multi-layer hierarchical model can be achieved by the proposed method. Finally, the paper shows the proposed technique which is used in a real-world application, where it is able to identify digit characters under various complex background images.

Application of Back-propagation Algorithm for the forecasting of Temperature and Humidity (온도 및 습도의 단기 예측에 있어서 역전파 알고리즘의 적용)

  • Jeong, Hyo-Joon;Hwang, Won-Tae;Suh, Kyung-Suk;Kim, Eun-Han;Han, Moon-Hee
    • Journal of Environmental Impact Assessment
    • /
    • v.12 no.4
    • /
    • pp.271-279
    • /
    • 2003
  • Temperature and humidity forecasting have been performed using artificial neural networks model(ANN). We composed ANN with multi-layer perceptron which is 2 input layers, 2 hidden layers and 1 output layer. Back propagation algorithm was used to train the ANN. 6 nodes and 12 nodes in the middle layers were appropriate to the temperature model for training. And 9 nodes and 6 nodes were also appropriate to the humidity model respectively. 90% of the all data was used learning set, and the extra 10% was used to model verification. In the case of temperature, average temperature before 15 minute and humidity at present constituted input layer, and temperature at present constituted out-layer and humidity model was vice versa. The sensitivity analysis revealed that previous value data contributed to forecasting target value than the other variable. Temperature was pseudo-linearly related to the previous 15 minute average value. We confirmed that ANN with multi-layer perceptron could support pollutant dispersion model by computing meterological data at real time.

Two Layer Multiquadric-Biharmonic Artificial Neural Network for Area Quasigeoid Surface Approximation with GPS-Levelling Data

  • Deng, Xingsheng;Wang, Xinzhou
    • Proceedings of the Korean Institute of Navigation and Port Research Conference
    • /
    • v.2
    • /
    • pp.101-106
    • /
    • 2006
  • The geoidal undulations are needed for determining the orthometric heights from the Global Positioning System GPS-derived ellipsoidal heights. There are several methods for geoidal undulation determination. The paper presents a method employing a simple architecture Two Layer Multiquadric-Biharmonic Artificial Neural Network (TLMB-ANN) to approximate an area of 4200 square kilometres quasigeoid surface with GPS-levelling data. Hardy’s Multiquadric-Biharmonic functions is used as the hidden layer neurons’ activation function and Levenberg-Marquardt algorithm is used to train the artificial neural network. In numerical examples five surfaces were compared: the gravimetric geometry hybrid quasigeoid, Support Vector Machine (SVM) model, Hybrid Fuzzy Neural Network (HFNN) model, Traditional Three Layer Artificial Neural Network (ANN) with tanh activation function and TLMB-ANN surface approximation. The effectiveness of TLMB-ANN surface approximation depends on the number of control points. If the number of well-distributed control points is sufficiently large, the results are similar with those obtained by gravity and geometry hybrid method. Importantly, TLMB-ANN surface approximation model possesses good extrapolation performance with high precision.

  • PDF

A Comparison of the Effects of Optimization Learning Rates using a Modified Learning Process for Generalized Neural Network (일반화 신경망의 개선된 학습 과정을 위한 최적화 신경망 학습률들의 효율성 비교)

  • Yoon, Yeochang;Lee, Sungduck
    • The Korean Journal of Applied Statistics
    • /
    • v.26 no.5
    • /
    • pp.847-856
    • /
    • 2013
  • We propose a modified learning process for generalized neural network using a learning algorithm by Liu et al. (2001). We consider the effect of initial weights, training results and learning errors using a modified learning process. We employ an incremental training procedure where training patterns are learned systematically. Our algorithm starts with a single training pattern and a single hidden layer neuron. During the course of neural network training, we try to escape from the local minimum by using a weight scaling technique. We allow the network to grow by adding a hidden layer neuron only after several consecutive failed attempts to escape from a local minimum. Our optimization procedure tends to make the network reach the error tolerance with no or little training after the addition of a hidden layer neuron. Simulation results with suitable initial weights indicate that the present constructive algorithm can obtain neural networks very close to minimal structures and that convergence to a solution in neural network training can be guaranteed. We tested these algorithms extensively with small training sets.

Improvement of Endoscopic Image using De-Interlacing Technique (De-Interlace 기법을 이용한 내시경 영상의 화질 개선)

  • 신동익;조민수;허수진
    • Journal of Biomedical Engineering Research
    • /
    • v.19 no.5
    • /
    • pp.469-476
    • /
    • 1998
  • In the case of acquisition and displaying medical Images such as ultrasonography and endoscopy on VGA monitor of PC system, image degradation of tear-drop appears through scan conversion. In this study, we compare several methods which can solve this degradation and implement the hardware system that resolves this problem in real-time with PC. It is possible to represent high quality image display and real-time processing and acquisition with specific de-interlacing device and PCI bridge on our hardware system. Image quality is improved remarkably on our hardware system. It is implemented as PC-based system, so acquiring, saving images and describing text comment on those images and PACS networking can be easily implemented.metabolism. All images were spatially normalized to MNI standard PET template and smoothed with 16mm FWHM Gaussian kernel using SPM96. Mean count in cerebral region was normalized. The VOls for 34 cerebral regions were previously defined on the standard template and 17 different counts of mirrored regions to hemispheric midline were extracted from spatially normalized images. A three-layer feed-forward error back-propagation neural network classifier with 7 input nodes and 3 output nodes was used. The network was trained to interpret metabolic patterns and produce identical diagnoses with those of expert viewers. The performance of the neural network was optimized by testing with 5~40 nodes in hidden layer. Randomly selected 40 images from each group were used to train the network and the remainders were used to test the learned network. The optimized neural network gave a maximum agreement rate of 80.3% with expert viewers. It used 20 hidden nodes and was trained for 1508 epochs. Also, neural network gave agreement rates of 75~80% with 10 or 30 nodes in hidden layer. We conclude that artificial neural network performed as well as human experts and could be potentially useful as clinical decision support tool for the localization of epileptogenic zones.

  • PDF

A Deep Learning Performance Comparison of R and Tensorflow (R과 텐서플로우 딥러닝 성능 비교)

  • Sung-Bong Jang
    • The Journal of the Convergence on Culture Technology
    • /
    • v.9 no.4
    • /
    • pp.487-494
    • /
    • 2023
  • In this study, performance comparison was performed on R and TensorFlow, which are free deep learning tools. In the experiment, six types of deep neural networks were built using each tool, and the neural networks were trained using the 10-year Korean temperature dataset. The number of nodes in the input layer of the constructed neural network was set to 10, the number of output layers was set to 5, and the hidden layer was set to 5, 10, and 20 to conduct experiments. The dataset includes 3600 temperature data collected from Gangnam-gu, Seoul from March 1, 2013 to March 29, 2023. For performance comparison, the future temperature was predicted for 5 days using the trained neural network, and the root mean square error (RMSE) value was measured using the predicted value and the actual value. Experiment results shows that when there was one hidden layer, the learning error of R was 0.04731176, and TensorFlow was measured at 0.06677193, and when there were two hidden layers, R was measured at 0.04782134 and TensorFlow was measured at 0.05799060. Overall, R was measured to have better performance. We tried to solve the difficulties in tool selection by providing quantitative performance information on the two tools to users who are new to machine learning.

Feasibility of Deep Learning Algorithms for Binary Classification Problems (이진 분류문제에서의 딥러닝 알고리즘의 활용 가능성 평가)

  • Kim, Kitae;Lee, Bomi;Kim, Jong Woo
    • Journal of Intelligence and Information Systems
    • /
    • v.23 no.1
    • /
    • pp.95-108
    • /
    • 2017
  • Recently, AlphaGo which is Bakuk (Go) artificial intelligence program by Google DeepMind, had a huge victory against Lee Sedol. Many people thought that machines would not be able to win a man in Go games because the number of paths to make a one move is more than the number of atoms in the universe unlike chess, but the result was the opposite to what people predicted. After the match, artificial intelligence technology was focused as a core technology of the fourth industrial revolution and attracted attentions from various application domains. Especially, deep learning technique have been attracted as a core artificial intelligence technology used in the AlphaGo algorithm. The deep learning technique is already being applied to many problems. Especially, it shows good performance in image recognition field. In addition, it shows good performance in high dimensional data area such as voice, image and natural language, which was difficult to get good performance using existing machine learning techniques. However, in contrast, it is difficult to find deep leaning researches on traditional business data and structured data analysis. In this study, we tried to find out whether the deep learning techniques have been studied so far can be used not only for the recognition of high dimensional data but also for the binary classification problem of traditional business data analysis such as customer churn analysis, marketing response prediction, and default prediction. And we compare the performance of the deep learning techniques with that of traditional artificial neural network models. The experimental data in the paper is the telemarketing response data of a bank in Portugal. It has input variables such as age, occupation, loan status, and the number of previous telemarketing and has a binary target variable that records whether the customer intends to open an account or not. In this study, to evaluate the possibility of utilization of deep learning algorithms and techniques in binary classification problem, we compared the performance of various models using CNN, LSTM algorithm and dropout, which are widely used algorithms and techniques in deep learning, with that of MLP models which is a traditional artificial neural network model. However, since all the network design alternatives can not be tested due to the nature of the artificial neural network, the experiment was conducted based on restricted settings on the number of hidden layers, the number of neurons in the hidden layer, the number of output data (filters), and the application conditions of the dropout technique. The F1 Score was used to evaluate the performance of models to show how well the models work to classify the interesting class instead of the overall accuracy. The detail methods for applying each deep learning technique in the experiment is as follows. The CNN algorithm is a method that reads adjacent values from a specific value and recognizes the features, but it does not matter how close the distance of each business data field is because each field is usually independent. In this experiment, we set the filter size of the CNN algorithm as the number of fields to learn the whole characteristics of the data at once, and added a hidden layer to make decision based on the additional features. For the model having two LSTM layers, the input direction of the second layer is put in reversed position with first layer in order to reduce the influence from the position of each field. In the case of the dropout technique, we set the neurons to disappear with a probability of 0.5 for each hidden layer. The experimental results show that the predicted model with the highest F1 score was the CNN model using the dropout technique, and the next best model was the MLP model with two hidden layers using the dropout technique. In this study, we were able to get some findings as the experiment had proceeded. First, models using dropout techniques have a slightly more conservative prediction than those without dropout techniques, and it generally shows better performance in classification. Second, CNN models show better classification performance than MLP models. This is interesting because it has shown good performance in binary classification problems which it rarely have been applied to, as well as in the fields where it's effectiveness has been proven. Third, the LSTM algorithm seems to be unsuitable for binary classification problems because the training time is too long compared to the performance improvement. From these results, we can confirm that some of the deep learning algorithms can be applied to solve business binary classification problems.

Analysis and Orange Utilization of Training Data and Basic Artificial Neural Network Development Results of Non-majors (비전공자 학부생의 훈련데이터와 기초 인공신경망 개발 결과 분석 및 Orange 활용)

  • Kyeong Hur
    • Journal of Practical Engineering Education
    • /
    • v.15 no.2
    • /
    • pp.381-388
    • /
    • 2023
  • Through artificial neural network education using spreadsheets, non-major undergraduate students can understand the operation principle of artificial neural networks and develop their own artificial neural network software. Here, training of the operation principle of artificial neural networks starts with the generation of training data and the assignment of correct answer labels. Then, the output value calculated from the firing and activation function of the artificial neuron, the parameters of the input layer, hidden layer, and output layer is learned. Finally, learning the process of calculating the error between the correct label of each initially defined training data and the output value calculated by the artificial neural network, and learning the process of calculating the parameters of the input layer, hidden layer, and output layer that minimize the total sum of squared errors. Training on the operation principles of artificial neural networks using a spreadsheet was conducted for undergraduate non-major students. And image training data and basic artificial neural network development results were collected. In this paper, we analyzed the results of collecting two types of training data and the corresponding artificial neural network SW with small 12-pixel images, and presented methods and execution results of using the collected training data for Orange machine learning model learning and analysis tools.

Deterministic Boltzmann Machine Based on Nonmonotonic Neuron Model (비단조 뉴런 모델을 이용한 결정론적 볼츠만 머신)

  • 강형원;박철영
    • Proceedings of the IEEK Conference
    • /
    • 2003.07d
    • /
    • pp.1553-1556
    • /
    • 2003
  • In this paper, We evaluate the learning ability of non-monotonic DBM(Deterministic Boltzmann Machine) network through numerical simulations. The simulation results show that the proposed system has higher performance than monotonic DBM network model. Non-monotonic DBM network also show an interesting result that network itself adjusts the number of hidden layer neurons. DBM network can be realized with fewer components than other neural network models. These results enhance the utilization of non-monotonic neurons in the large scale integration of neuro-chips.

  • PDF