• Title/Summary/Keyword: ReLU layer

Search Result 20, Processing Time 0.03 seconds

A Study on the Influence of Ceramic Coating on Characteristics of Friction and Abrasion of Aluminum Alloy(7075 T6) Used in Mechanical Casting (세라믹 코팅이 기계 주조용 알루미늄합금(7075 T6)의 마찰ㆍ마모특성에 미치는 영향에 관한 연구)

  • Lyu, Sung-Ki;Jung, Kwang-Jo;Lu, Long
    • Journal of the Korean Society of Safety
    • /
    • v.18 no.3
    • /
    • pp.14-21
    • /
    • 2003
  • This study deals with the influence of ceramic coating on characteristics of friction and abrasion of aluminum allot(7075 T6) used in mechanical casting. In this research, frictional wear characteristic of ceramic coating materials such as $A1_2O_3$, $Si_3N_4$, SiC was investigated using aluminum alloy(7075 T6) and stainless 403 cast iron under room temperature and normal air pressure. The coating layer was observed using SEM. The conclusions are as follows: 1) Friction coefficients of $A1_2O_3$, SiC and $Si_3N_4$ are obtained 0.63 0.56 and 0.54 respectively. 2) Abrasion resistance of stainless 403 cast iron with $Si_3N_4$ is the best among the ceramic coating materials. 3) Abrasion mechanism of aluminum alloy(7075 T6) coaled with ceramic material and stainless 403 cast iron is caused by brittle fracture. 4) Coating the ceramic material on the aluminum alloy(7075 T6) can effectively increase the antiwear, impact properties, and corrosion resistance.

Animal Fur Recognition Algorithm Based on Feature Fusion Network

  • Liu, Peng;Lei, Tao;Xiang, Qian;Wang, Zexuan;Wang, Jiwei
    • Journal of Multimedia Information System
    • /
    • v.9 no.1
    • /
    • pp.1-10
    • /
    • 2022
  • China is a big country in animal fur industry. The total production and consumption of fur are increasing year by year. However, the recognition of fur in the fur production process still mainly relies on the visual identification of skilled workers, and the stability and consistency of products cannot be guaranteed. In response to this problem, this paper proposes a feature fusion-based animal fur recognition network on the basis of typical convolutional neural network structure, relying on rapidly developing deep learning techniques. This network superimposes texture feature - the most prominent feature of fur image - into the channel dimension of input image. The output feature map of the first layer convolution is inverted to obtain the inverted feature map and concat it into the original output feature map, then Leaky ReLU is used for activation, which makes full use of the texture information of fur image and the inverted feature information. Experimental results show that the algorithm improves the recognition accuracy by 9.08% on Fur_Recognition dataset and 6.41% on CIFAR-10 dataset. The algorithm in this paper can change the current situation that fur recognition relies on manual visual method to classify, and can lay foundation for improving the efficiency of fur production technology.

Study on the Reconstruction of Pressure Field in Sloshing Simulation Using Super-Resolution Convolutional Neural Network (심층학습 기반 초해상화 기법을 이용한 슬로싱 압력장 복원에 관한 연구)

  • Kim, Hyo Ju;Yang, Donghun;Park, Jung Yoon;Hwang, Myunggwon;Lee, Sang Bong
    • Journal of the Society of Naval Architects of Korea
    • /
    • v.59 no.2
    • /
    • pp.72-79
    • /
    • 2022
  • Deep-learning-based Super-Resolution (SR) methods were evaluated to reconstruct pressure fields with a high resolution from low-resolution images taken from a coarse grid simulation. In addition to a canonical SRCNN(super-resolution convolutional neural network) model, two modified models from SRCNN, adding an activation function (ReLU or Sigmoid function) to the output layer, were considered in the present study. High resolution images obtained by three models were more vivid and reliable qualitatively, compared with a conventional super-resolution method of bicubic interpolation. A quantitative comparison of statistical similarity showed that SRCNN model with Sigmoid function achieved best performance with less dependency on original resolution of input images.

A study on discharge estimation for the event using a deep learning algorithm (딥러닝 알고리즘을 이용한 강우 발생시의 유량 추정에 관한 연구)

  • Song, Chul Min
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2021.06a
    • /
    • pp.246-246
    • /
    • 2021
  • 본 연구는 강우 발생시 유량을 추정하는 것에 목적이 있다. 이를 위해 본 연구는 선행연구의 모형 개발방법론에서 벗어나 딥러닝 알고리즘 중 하나인 합성곱 신경망 (convolution neural network)과 수문학적 이미지 (hydrological image)를 이용하여 강우 발생시 유량을 추정하였다. 합성곱 신경망은 일반적으로 분류 문제 (classification)을 해결하기 위한 목적으로 개발되었기 때문에 불특정 연속변수인 유량을 모의하기에는 적합하지 않다. 이를 위해 본 연구에서는 합성곱 신경망의 완전 연결층 (Fully connected layer)를 개선하여 연속변수를 모의할 수 있도록 개선하였다. 대부분 합성곱 신경망은 RGB (red, green, blue) 사진 (photograph)을 이용하여 해당 사진이 나타내는 것을 예측하는 목적으로 사용하지만, 본 연구의 경우 일반 RGB 사진을 이용하여 유출량을 예측하는 것은 경험적 모형의 전제(독립변수와 종속변수의 관계)를 무너뜨리는 결과를 초래할 수 있다. 이를 위해 본 연구에서는 임의의 유역에 대해 2차원 공간에서 무차원의 수문학적 속성을 갖는 grid의 집합으로 정의되는 수문학적 이미지는 입력자료로 활용했다. 합성곱 신경망의 구조는 Convolution Layer와 Pulling Layer가 5회 반복하는 구조로 설정하고, 이후 Flatten Layer, 2개의 Dense Layer, 1개의 Batch Normalization Layer를 배열하고, 다시 1개의 Dense Layer가 이어지는 구조로 설계하였다. 마지막 Dense Layer의 활성화 함수는 분류모형에 이용되는 softmax 또는 sigmoid 함수를 대신하여 회귀모형에서 자주 사용되는 Linear 함수로 설정하였다. 이와 함께 각 층의 활성화 함수는 정규화 선형함수 (ReLu)를 이용하였으며, 모형의 학습 평가 및 검정을 판단하기 위해 MSE 및 MAE를 사용했다. 또한, 모형평가는 NSE와 RMSE를 이용하였다. 그 결과, 모형의 학습 평가에 대한 MSE는 11.629.8 m3/s에서 118.6 m3/s로, MAE는 25.4 m3/s에서 4.7 m3/s로 감소하였으며, 모형의 검정에 대한 MSE는 1,997.9 m3/s에서 527.9 m3/s로, MAE는 21.5 m3/s에서 9.4 m3/s로 감소한 것으로 나타났다. 또한, 모형평가를 위한 NSE는 0.7, RMSE는 27.0 m3/s로 나타나, 본 연구의 모형은 양호(moderate)한 것으로 판단하였다. 이에, 본 연구를 통해 제시된 방법론에 기반을 두어 CNN 모형 구조의 확장과 수문학적 이미지의 개선 또는 새로운 이미지 개발 등을 추진할 경우 모형의 예측 성능이 향상될 수 있는 여지가 있으며, 원격탐사 분야나, 위성 영상을 이용한 전 지구적 또는 광역 단위의 실시간 유량 모의 분야 등으로의 응용이 가능할 것으로 기대된다.

  • PDF

A study on pollutant loads prediction using a convolution neural networks (합성곱 신경망을 이용한 오염부하량 예측에 관한 연구)

  • Song, Chul Min
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2021.06a
    • /
    • pp.444-444
    • /
    • 2021
  • 하천의 오염부하량 관리 계획은 지속적인 모니터링을 통한 자료 구축과 모형을 이용한 예측결과를 기반으로 수립된다. 하천의 모니터링과 예측 분석은 많은 예산과 인력 등이 필요하나, 정부의 담당 공무원 수는 극히 부족한 상황이 일반적이다. 이에 정부는 전문가에게 관련 용역을 의뢰하지만, 한국과 같이 지형이 복잡한 지역에서의 오염부하량 배출 특성은 각각 다르게 나타나기 때문에 많은 예산 소모가 발생 된다. 이를 개선하고자, 본 연구는 합성곱 신경망 (convolution neural network)과 수문학적 이미지 (hydrological image)를 이용하여 강우 발생시 BOD 및 총인의 부하량 예측 모형을 개발하였다. 합성곱 신경망의 입력자료는 일반적으로 RGB (red, green, bule) 사진을 이용하는데, 이를 그래도 오염부하량 예측에 활용하는 것은 경험적 모형의 전제(독립변수와 종속변수의 관계)를 무너뜨리는 결과를 초래할 수 있다. 이에, 본 연구에서는 오염부하량이 수문학적 조건과 토지이용 등의 변수에 의해 결정된다는 인과관계를 만족시키고자 수문학적 속성이 내재된 수문학적 이미지를 합성곱 신경망의 훈련자료로 사용하였다. 수문학적 이미지는 임의의 유역에 대해 2차원 공간에서 무차원의 수문학적 속성을 갖는 grid의 집합으로 정의되는데, 여기서 각 grid의 수문학적 속성은 SCS 토양보존국(soil conservation service, SCS)에서 발표한 수문학적 토양피복형수 (curve number, CN)를 이용하여 산출한다. 합성곱 신경망의 구조는 2개의 Convolution Layer와 1개의 Pulling Layer가 5회 반복하는 구조로 설정하고, 1개의 Flatten Layer, 3개의 Dense Layer, 1개의 Batch Normalization Layer를 배열하고, 마지막으로 1개의 Dense Layer가 연결되는 구조로 설계하였다. 이와 함께, 각 층의 활성화 함수는 정규화 선형함수 (ReLu)로, 마지막 Dense Layer의 활성화 함수는 연속변수가 도출될 수 있도록 회귀모형에서 자주 사용되는 Linear 함수로 설정하였다. 연구의 대상지역은 경기도 가평군 조종천 유역으로 선정하였고, 연구기간은 2010년 1월 1일부터 2019년 12월 31일까지로, 2010년부터 2016년까지의 자료는 모형의 학습에, 2017년부터 2019년까지의 자료는 모형의 성능평가에 활용하였다. 모형의 예측 성능은 모형효율계수 (NSE), 평균제곱근오차(RMSE) 및 평균절대백분율오차(MAPE)를 이용하여 평가하였다. 그 결과, BOD 부하량에 대한 NSE는 0.9, RMSE는 1031.1 kg/day, MAPE는 11.5%로 나타났으며, 총인 부하량에 대한 NSE는 0.9, RMSE는 53.6 kg/day, MAPE는 17.9%로 나타나 본 연구의 모형은 우수(good)한 것으로 판단하였다. 이에, 본 연구의 모형은 일반 ANN 모형을 이용한 선행연구와는 달리 2차원 공간정보를 반영하여 오염부하량 모의가 가능했으며, 제한적인 입력자료를 이용하여 간편한 모델링이 가능하다는 장점을 나타냈다. 이를 통해 정부의 물관리 정책을 위한 의사결정 및 부족한 물관리 분야의 행정력에 도움이 될 것으로 생각된다.

  • PDF

Comparison of Gradient Descent for Deep Learning (딥러닝을 위한 경사하강법 비교)

  • Kang, Min-Jae
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.21 no.2
    • /
    • pp.189-194
    • /
    • 2020
  • This paper analyzes the gradient descent method, which is the one most used for learning neural networks. Learning means updating a parameter so the loss function is at its minimum. The loss function quantifies the difference between actual and predicted values. The gradient descent method uses the slope of the loss function to update the parameter to minimize error, and is currently used in libraries that provide the best deep learning algorithms. However, these algorithms are provided in the form of a black box, making it difficult to identify the advantages and disadvantages of various gradient descent methods. This paper analyzes the characteristics of the stochastic gradient descent method, the momentum method, the AdaGrad method, and the Adadelta method, which are currently used gradient descent methods. The experimental data used a modified National Institute of Standards and Technology (MNIST) data set that is widely used to verify neural networks. The hidden layer consists of two layers: the first with 500 neurons, and the second with 300. The activation function of the output layer is the softmax function, and the rectified linear unit function is used for the remaining input and hidden layers. The loss function uses cross-entropy error.

Optimization Of Water Quality Prediction Model In Daechong Reservoir, Based On Multiple Layer Perceptron (다층 퍼셉트론을 기반으로 한 대청호 수질 예측 모델 최적화)

  • Lee, Hankyu;Kim, Jin Hui;Byeon, Seohyeon;Park, Kangdong;Shin, Jae-ki;Park, Yongeun
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2022.05a
    • /
    • pp.43-43
    • /
    • 2022
  • 유해 조류 대발생은 전국 각지의 인공호소나 하천에서 다발적으로 발생하며, 경관을 해치고 수질을 오염시키는 등 수자원에 부정적인 영향을 미친다. 본 연구에서는 인공호소에서 발생하는 유해 조류 대발생을 예측하기 위해 심층학습 기법을 이용하여 예측 모델을 개발하고자 하였다. 대상 지점은 대청호의 추동 지점으로 선정하였다. 대청호는 금강유역 중류에 위치한 댐으로, 약 150만명에 달하는 급수 인구수를 유지 중이기에 유해 남조 대발생 관리가 매우 중요한 장소이다. 학습용 데이터 구축은 대청호의 2011년 1월부터 2019년 12월까지 측정된 수질, 기상, 수문 자료를 입력 자료를 이용하였다. 수질 예측 모델의 구조는 다중 레이어 퍼셉트론(Multiple Layer Perceptron; MLP)으로, 입력과 한 개 이상의 은닉층, 그리고 출력층으로 구성된 인공신경망이다. 본 연구에서는 인공신경망의 은닉층 개수(1~3개)와 각각의 레이어에 적용되는 은닉 노드 개수(11~30개), 활성함수 5종(Linear, sigmoid, hyperbolic tangent, Rectified Linear Unit, Exponential Linear Unit)을 각각 하이퍼파라미터로 정하고, 모델의 성능을 최대로 발휘할 수 있는 조건을 찾고자 하였다. 하이퍼파라미터 최적화 도구는 Tensorflow에서 배포하는 Keras Tuner를 사용하였다. 모델은 총 3000 학습 epoch 가 진행되는 동안 최적의 가중치를 계산하도록 설계하였고, 이 결과를 매 반복마다 저장장치에 기록하였다. 모델 성능의 타당성은 예측과 실측 데이터 간의 상관관계를 R2, NSE, RMSE를 통해 산출하여 검증하였다. 모델 최적화 결과, 적합한 하이퍼파라미터는 최적화 횟수 총 300회에서 256 번째 반복 결과인 은닉층 개수 3개, 은닉 노드 수 각각 25개, 22개, 14개가 가장 적합하였고, 이에 따른 활성함수는 ELU, ReLU, Hyperbolic tangent, Linear 순서대로 사용되었다. 최적화된 하이퍼파라미터를 이용하여 모델 학습 및 검증을 수행한 결과, R2는 학습 0.68, 검증 0.61이었고 NSE는 학습 0.85, 검증 0.81, RMSE는 학습 0.82, 검증 0.92로 나타났다.

  • PDF

Quality grading of Hanwoo (Korean native cattle breed) sub-images using convolutional neural network

  • Kwon, Kyung-Do;Lee, Ahyeong;Lim, Jongkuk;Cho, Soohyun;Lee, Wanghee;Cho, Byoung-Kwan;Seo, Youngwook
    • Korean Journal of Agricultural Science
    • /
    • v.47 no.4
    • /
    • pp.1109-1122
    • /
    • 2020
  • The aim of this study was to develop a marbling classification and prediction model using small parts of sirloin images based on a deep learning algorithm, namely, a convolutional neural network (CNN). Samples were purchased from a commercial slaughterhouse in Korea, images for each grade were acquired, and the total images (n = 500) were assigned according to their grade number: 1++, 1+, 1, and both 2 & 3. The image acquisition system consists of a DSLR camera with a polarization filter to remove diffusive reflectance and two light sources (55 W). To correct the distorted original images, a radial correction algorithm was implemented. Color images of sirloins of Hanwoo (mixed with feeder cattle, steer, and calf) were divided and sub-images with image sizes of 161 × 161 were made to train the marbling prediction model. In this study, the convolutional neural network (CNN) has four convolution layers and yields prediction results in accordance with marbling grades (1++, 1+, 1, and 2&3). Every single layer uses a rectified linear unit (ReLU) function as an activation function and max-pooling is used for extracting the edge between fat and muscle and reducing the variance of the data. Prediction accuracy was measured using an accuracy and kappa coefficient from a confusion matrix. We summed the prediction of sub-images and determined the total average prediction accuracy. Training accuracy was 100% and the test accuracy was 86%, indicating comparably good performance using the CNN. This study provides classification potential for predicting the marbling grade using color images and a convolutional neural network algorithm.

Predicting blast-induced ground vibrations at limestone quarry from artificial neural network optimized by randomized and grid search cross-validation, and comparative analyses with blast vibration predictor models

  • Salman Ihsan;Shahab Saqib;Hafiz Muhammad Awais Rashid;Fawad S. Niazi;Mohsin Usman Qureshi
    • Geomechanics and Engineering
    • /
    • v.35 no.2
    • /
    • pp.121-133
    • /
    • 2023
  • The demand for cement and limestone crushed materials has increased many folds due to the tremendous increase in construction activities in Pakistan during the past few decades. The number of cement production industries has increased correspondingly, and so the rock-blasting operations at the limestone quarry sites. However, the safety procedures warranted at these sites for the blast-induced ground vibrations (BIGV) have not been adequately developed and/or implemented. Proper prediction and monitoring of BIGV are necessary to ensure the safety of structures in the vicinity of these quarry sites. In this paper, an attempt has been made to predict BIGV using artificial neural network (ANN) at three selected limestone quarries of Pakistan. The ANN has been developed in Python using Keras with sequential model and dense layers. The hyper parameters and neurons in each of the activation layers has been optimized using randomized and grid search method. The input parameters for the model include distance, a maximum charge per delay (MCPD), depth of hole, burden, spacing, and number of blast holes, whereas, peak particle velocity (PPV) is taken as the only output parameter. A total of 110 blast vibrations datasets were recorded from three different limestone quarries. The dataset has been divided into 85% for neural network training, and 15% for testing of the network. A five-layer ANN is trained with Rectified Linear Unit (ReLU) activation function, Adam optimization algorithm with a learning rate of 0.001, and batch size of 32 with the topology of 6-32-32-256-1. The blast datasets were utilized to compare the performance of ANN, multivariate regression analysis (MVRA), and empirical predictors. The performance was evaluated using the coefficient of determination (R2), mean absolute error (MAE), mean squared error (MSE), mean absolute percentage error (MAPE), and root mean squared error (RMSE)for predicted and measured PPV. To determine the relative influence of each parameter on the PPV, sensitivity analyses were performed for all input parameters. The analyses reveal that ANN performs superior than MVRA and other empirical predictors, andthat83% PPV is affected by distance and MCPD while hole depth, number of blast holes, burden and spacing contribute for the remaining 17%. This research provides valuable insights into improving safety measures and ensuring the structural integrity of buildings near limestone quarry sites.

A Time Series Graph based Convolutional Neural Network Model for Effective Input Variable Pattern Learning : Application to the Prediction of Stock Market (효과적인 입력변수 패턴 학습을 위한 시계열 그래프 기반 합성곱 신경망 모형: 주식시장 예측에의 응용)

  • Lee, Mo-Se;Ahn, Hyunchul
    • Journal of Intelligence and Information Systems
    • /
    • v.24 no.1
    • /
    • pp.167-181
    • /
    • 2018
  • Over the past decade, deep learning has been in spotlight among various machine learning algorithms. In particular, CNN(Convolutional Neural Network), which is known as the effective solution for recognizing and classifying images or voices, has been popularly applied to classification and prediction problems. In this study, we investigate the way to apply CNN in business problem solving. Specifically, this study propose to apply CNN to stock market prediction, one of the most challenging tasks in the machine learning research. As mentioned, CNN has strength in interpreting images. Thus, the model proposed in this study adopts CNN as the binary classifier that predicts stock market direction (upward or downward) by using time series graphs as its inputs. That is, our proposal is to build a machine learning algorithm that mimics an experts called 'technical analysts' who examine the graph of past price movement, and predict future financial price movements. Our proposed model named 'CNN-FG(Convolutional Neural Network using Fluctuation Graph)' consists of five steps. In the first step, it divides the dataset into the intervals of 5 days. And then, it creates time series graphs for the divided dataset in step 2. The size of the image in which the graph is drawn is $40(pixels){\times}40(pixels)$, and the graph of each independent variable was drawn using different colors. In step 3, the model converts the images into the matrices. Each image is converted into the combination of three matrices in order to express the value of the color using R(red), G(green), and B(blue) scale. In the next step, it splits the dataset of the graph images into training and validation datasets. We used 80% of the total dataset as the training dataset, and the remaining 20% as the validation dataset. And then, CNN classifiers are trained using the images of training dataset in the final step. Regarding the parameters of CNN-FG, we adopted two convolution filters ($5{\times}5{\times}6$ and $5{\times}5{\times}9$) in the convolution layer. In the pooling layer, $2{\times}2$ max pooling filter was used. The numbers of the nodes in two hidden layers were set to, respectively, 900 and 32, and the number of the nodes in the output layer was set to 2(one is for the prediction of upward trend, and the other one is for downward trend). Activation functions for the convolution layer and the hidden layer were set to ReLU(Rectified Linear Unit), and one for the output layer set to Softmax function. To validate our model - CNN-FG, we applied it to the prediction of KOSPI200 for 2,026 days in eight years (from 2009 to 2016). To match the proportions of the two groups in the independent variable (i.e. tomorrow's stock market movement), we selected 1,950 samples by applying random sampling. Finally, we built the training dataset using 80% of the total dataset (1,560 samples), and the validation dataset using 20% (390 samples). The dependent variables of the experimental dataset included twelve technical indicators popularly been used in the previous studies. They include Stochastic %K, Stochastic %D, Momentum, ROC(rate of change), LW %R(Larry William's %R), A/D oscillator(accumulation/distribution oscillator), OSCP(price oscillator), CCI(commodity channel index), and so on. To confirm the superiority of CNN-FG, we compared its prediction accuracy with the ones of other classification models. Experimental results showed that CNN-FG outperforms LOGIT(logistic regression), ANN(artificial neural network), and SVM(support vector machine) with the statistical significance. These empirical results imply that converting time series business data into graphs and building CNN-based classification models using these graphs can be effective from the perspective of prediction accuracy. Thus, this paper sheds a light on how to apply deep learning techniques to the domain of business problem solving.