• Title/Summary/Keyword: ReLU layer

Search Result 20, Processing Time 0.031 seconds

Deep Learning System based on Morphological Neural Network (몰포러지 신경망 기반 딥러닝 시스템)

  • Choi, Jong-Ho
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.12 no.1
    • /
    • pp.92-98
    • /
    • 2019
  • In this paper, we propose a deep learning system based on morphological neural network(MNN). The deep learning layers are morphological operation layer, pooling layer, ReLU layer, and the fully connected layer. The operations used in morphological layer are erosion, dilation, and edge detection, etc. Unlike CNN, the number of hidden layers and kernels applied to each layer is limited in MNN. Because of the reduction of processing time and utility of VLSI chip design, it is possible to apply MNN to various mobile embedded systems. MNN performs the edge and shape detection operations with a limited number of kernels. Through experiments using database images, it is confirmed that MNN can be used as a deep learning system and its performance.

Design of new CNN structure with internal FC layer (내부 FC층을 갖는 새로운 CNN 구조의 설계)

  • Park, Hee-mun;Park, Sung-chan;Hwang, Kwang-bok;Choi, Young-kiu;Park, Jin-hyun
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2018.05a
    • /
    • pp.466-467
    • /
    • 2018
  • Recently, artificial intelligence has been applied to various fields such as image recognition, image recognition speech recognition, and natural language processing, and interest in Deep Learning technology is increasing. Many researches on Convolutional Neural Network(CNN), which is one of the most representative algorithms among Deep Learning, have strong advantages in image recognition and classification and are widely used in various fields. In this paper, we propose a new network structure that transforms the general CNN structure. A typical CNN structure consists of a convolution layer, ReLU layer, and a pooling layer. Therefore in this paper, We intend to construct a new network by adding fully connected layer inside a general CNN structure. This modification is intended to increase the learning and accuracy of the convoluted image by including the generalization which is an advantage of the neural network.

  • PDF

Development of Deep Learning Structure to Improve Quality of Polygonal Containers (다각형 용기의 품질 향상을 위한 딥러닝 구조 개발)

  • Yoon, Suk-Moon;Lee, Seung-Ho
    • Journal of IKEEE
    • /
    • v.25 no.3
    • /
    • pp.493-500
    • /
    • 2021
  • In this paper, we propose the development of deep learning structure to improve quality of polygonal containers. The deep learning structure consists of a convolution layer, a bottleneck layer, a fully connect layer, and a softmax layer. The convolution layer is a layer that obtains a feature image by performing a convolution 3x3 operation on the input image or the feature image of the previous layer with several feature filters. The bottleneck layer selects only the optimal features among the features on the feature image extracted through the convolution layer, reduces the channel to a convolution 1x1 ReLU, and performs a convolution 3x3 ReLU. The global average pooling operation performed after going through the bottleneck layer reduces the size of the feature image by selecting only the optimal features among the features of the feature image extracted through the convolution layer. The fully connect layer outputs the output data through 6 fully connect layers. The softmax layer multiplies and multiplies the value between the value of the input layer node and the target node to be calculated, and converts it into a value between 0 and 1 through an activation function. After the learning is completed, the recognition process classifies non-circular glass bottles by performing image acquisition using a camera, measuring position detection, and non-circular glass bottle classification using deep learning as in the learning process. In order to evaluate the performance of the deep learning structure to improve quality of polygonal containers, as a result of an experiment at an authorized testing institute, it was calculated to be at the same level as the world's highest level with 99% good/defective discrimination accuracy. Inspection time averaged 1.7 seconds, which was calculated within the operating time standards of production processes using non-circular machine vision systems. Therefore, the effectiveness of the performance of the deep learning structure to improve quality of polygonal containers proposed in this paper was proven.

Comparison of Artificial Neural Network Model Capability for Runoff Estimation about Activation Functions (활성화 함수에 따른 유출량 산정 인공신경망 모형의 성능 비교)

  • Kim, Maga;Choi, Jin-Yong;Bang, Jehong;Yoon, Pureun;Kim, Kwihoon
    • Journal of The Korean Society of Agricultural Engineers
    • /
    • v.63 no.1
    • /
    • pp.103-116
    • /
    • 2021
  • Analysis of runoff is substantial for effective water management in the watershed. Runoff occurs by reaction of a watershed to the rainfall and has non-linearity and uncertainty due to the complex relation of weather and watershed factors. ANN (Artificial Neural Network), which learns from the data, is one of the machine learning technique known as a proper model to interpret non-linear data. The performance of ANN is affected by the ANN's structure, the number of hidden layer nodes, learning rate, and activation function. Especially, the activation function has a role to deliver the information entered and decides the way of making output. Therefore, It is important to apply appropriate activation functions according to the problem to solve. In this paper, ANN models were constructed to estimate runoff with different activation functions and each model was compared and evaluated. Sigmoid, Hyperbolic tangent, ReLU (Rectified Linear Unit), ELU (Exponential Linear Unit) functions were applied to the hidden layer, and Identity, ReLU, Softplus functions applied to the output layer. The statistical parameters including coefficient of determination, NSE (Nash and Sutcliffe Efficiency), NSEln (modified NSE), and PBIAS (Percent BIAS) were utilized to evaluate the ANN models. From the result, applications of Hyperbolic tangent function and ELU function to the hidden layer and Identity function to the output layer show competent performance rather than other functions which demonstrated the function selection in the ANN structure can affect the performance of ANN.

Discernment of Android User Interaction Data Distribution Using Deep Learning

  • Ho, Jun-Won
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.14 no.3
    • /
    • pp.143-148
    • /
    • 2022
  • In this paper, we employ deep neural network (DNN) to discern Android user interaction data distribution from artificial data distribution. We utilize real Android user interaction trace dataset collected from [1] to evaluate our DNN design. In particular, we use sequential model with 4 dense hidden layers and 1 dense output layer in TensorFlow and Keras. We also deploy sigmoid activation function for a dense output layer with 1 neuron and ReLU activation function for each dense hidden layer with 32 neurons. Our evaluation shows that our DNN design fulfills high test accuracy of at least 0.9955 and low test loss of at most 0.0116 in all cases of artificial data distributions.

Optimal Algorithm and Number of Neurons in Deep Learning (딥러닝 학습에서 최적의 알고리즘과 뉴론수 탐색)

  • Jang, Ha-Young;You, Eun-Kyung;Kim, Hyeock-Jin
    • Journal of Digital Convergence
    • /
    • v.20 no.4
    • /
    • pp.389-396
    • /
    • 2022
  • Deep Learning is based on a perceptron, and is currently being used in various fields such as image recognition, voice recognition, object detection, and drug development. Accordingly, a variety of learning algorithms have been proposed, and the number of neurons constituting a neural network varies greatly among researchers. This study analyzed the learning characteristics according to the number of neurons of the currently used SGD, momentum methods, AdaGrad, RMSProp, and Adam methods. To this end, a neural network was constructed with one input layer, three hidden layers, and one output layer. ReLU was applied to the activation function, cross entropy error (CEE) was applied to the loss function, and MNIST was used for the experimental dataset. As a result, it was concluded that the number of neurons 100-300, the algorithm Adam, and the number of learning (iteraction) 200 would be the most efficient in deep learning learning. This study will provide implications for the algorithm to be developed and the reference value of the number of neurons given new learning data in the future.

Efficiency of an SCM415 Alloy Surface Layer Implanted with Nitrogen Ions by Plasma Source Ion Implantation

  • Lyu, Sung-Ki;He, Hui-Bo;Lu, Long;Youn, Il-Joong
    • International Journal of Precision Engineering and Manufacturing
    • /
    • v.7 no.4
    • /
    • pp.47-50
    • /
    • 2006
  • SCM415 alloy was implanted with nitrogen ions using plasma source ion implantation (PSII), at a dose range of $1{\times}10^{17}\;to\;6{\times}10^{17}\;N^+cm^{-2}$ Auger electron spectrometry (AES) was used to investigate the depth profile of the implanted layer. Friction and wear tests were carried out on a block-on-ring wear tester. Scanning electron microscopy (SEM) was used to observe the micro-morphology of the worn surface. The results revealed that after being implanted with nitrogen ions, the frictional coefficient of the surface layer decreased, and the wear resistance increased with the nitrogen dose. The tribological mechanism was mainly adhesive, and the adhesive wear tended to become weaker oxidative wear with the increase in the nitrogen dose. The effects were mainly attributed to the formation of a hard nitride precipitate and a supersaturated solid solution of nitrogen in the surface layer.

A Study on the Friction and Wear Characteristics of C-N Coated Spur Gear (C-N 코팅 스퍼기어의 마찰${\cdot}$마모 특성에 관한 연구)

  • Lu Long;Lyu Sung-ki
    • Proceedings of the Korean Society of Tribologists and Lubrication Engineers Conference
    • /
    • 2004.11a
    • /
    • pp.41-46
    • /
    • 2004
  • This study deals with the friction and wear characteristics of C-N coated spur gear. The PSII apparatus was built and a SCM415 test piece and test gear with steel substrate was treated with carbon nitrogen by this apparatus. The composition and structure of the surface layer were analyzed and compared with that of PVD coated TiN layer. It was found that both of friction coefficients of C-N coating and TiN coaling decreased with increasing load, however, C-N coating showed relatively lower friction coefficient than that of TiN coating. We was investigated the effect of C-N coating on hardness, friction and wear. The TiN coated gear showed more serious friction phenomena than that of C-N coated gear. It was considered that coating of TiN, which was conducted at a vacuum chamber at about $500^{\circ}C$ results in a tempering of base material that causes microstructure change, which in turn resulted in decreasing of hardness. The C-N coated gear and pinion had higher wear resistance that of TiN coated gear and pinion. C-N coating significantly improved the friction and wear resistance of the gear.

  • PDF

Investigation of the Super-resolution Algorithm for the Prediction of Periodontal Disease in Dental X-ray Radiography (치주질환 예측을 위한 치과 X-선 영상에서의 초해상화 알고리즘 적용 가능성 연구)

  • Kim, Han-Na
    • Journal of the Korean Society of Radiology
    • /
    • v.15 no.2
    • /
    • pp.153-158
    • /
    • 2021
  • X-ray image analysis is a very important field to improve the early diagnosis rate and prediction accuracy of periodontal disease. Research on the development and application of artificial intelligence-based algorithms to improve the quality of such dental X-ray images is being widely conducted worldwide. Thus, the aim of this study was to design a super-resolution algorithm for predicting periodontal disease and to evaluate its applicability in dental X-ray images. The super-resolution algorithm was constructed based on the convolution layer and ReLU, and an image obtained by up-sampling a low-resolution image by 2 times was used as an input data. Also, 1,500 dental X-ray data used for deep learning training were used. Quantitative evaluation of images used root mean square error and structural similarity, which are factors that can measure similarity through comparison of two images. In addition, the recently developed no-reference based natural image quality evaluator and blind/referenceless image spatial quality evaluator were additionally analyzed. According to the results, we confirmed that the average similarity and no-reference-based evaluation values were improved by 1.86 and 2.14 times, respectively, compared to the existing bicubic-based upsampling method when the proposed method was used. In conclusion, the super-resolution algorithm for predicting periodontal disease proved useful in dental X-ray images, and it is expected to be highly applicable in various fields in the future.

Full-scale tests and analytical model of the Teflon-based lead rubber isolation bearings

  • Wang, Lu;Oua, Jin;Liu, Weiqing;Wang, Shuguang
    • Structural Engineering and Mechanics
    • /
    • v.48 no.6
    • /
    • pp.809-822
    • /
    • 2013
  • Base isolation is widely used in seismic resisting buildings due to its low construction cost, high reliability, mature theory and convenient usage. However, it is difficult to design the isolation layer in high-rise buildings using the available bearings because high-rise buildings are characterized with long period, low horizontal stiffness, and complex re-distribution of the internal forces under earthquake loads etc. In this paper, a simple and innovative isolation bearing, named Teflon-based lead rubber isolation bearing, is developed to address the mentioned problems. The Teflon-based lead rubber isolation bearing consists of friction material and lead rubber isolation bearing. Hence, it integrates advantages of friction bearings and lead rubber isolation bearings so that improves the stability of base isolation system. An experimental study was conducted to validate the effectiveness of this new bearing. The effects of vertical loading, displacement amplitude and loading frequency on the force-displacement relationship and energy dissipation capacity of the Teflon-based lead rubber isolation bearing were studied. An analytical model was also proposed to predict the force-displacement relationship of the new bearing. Comparison of analytical and experimental results showed that the analytical model can accurately predict the force-displacement relationship and elastic shear deflection of the Teflon-based lead rubber isolation bearings.