• 한국멀티미디어학회논문지
• /
• 제21권10호
• /
• pp.1142-1149
• /
• 2018

Recently, computer vision application is increasing by using CNN which is one of the deep learning algorithms. However, CNN does not provide perfect classification performance due to gradient vanishing problem. Most of CNN algorithms use an activation function called ReLU to mitigate the gradient vanishing problem. In this study, four activation functions that can replace ReLU were applied to four different structural networks. Experimental results show that ReLU has the lowest performance in accuracy, loss rate, and speed of initial learning convergence from 20 experiments. It is concluded that the optimal activation function varied from network to network but the four activation functions were higher than ReLU.

• 한국농공학회논문집
• /
• 제63권1호
• /
• pp.103-116
• /
• 2021

Analysis of runoff is substantial for effective water management in the watershed. Runoff occurs by reaction of a watershed to the rainfall and has non-linearity and uncertainty due to the complex relation of weather and watershed factors. ANN (Artificial Neural Network), which learns from the data, is one of the machine learning technique known as a proper model to interpret non-linear data. The performance of ANN is affected by the ANN's structure, the number of hidden layer nodes, learning rate, and activation function. Especially, the activation function has a role to deliver the information entered and decides the way of making output. Therefore, It is important to apply appropriate activation functions according to the problem to solve. In this paper, ANN models were constructed to estimate runoff with different activation functions and each model was compared and evaluated. Sigmoid, Hyperbolic tangent, ReLU (Rectified Linear Unit), ELU (Exponential Linear Unit) functions were applied to the hidden layer, and Identity, ReLU, Softplus functions applied to the output layer. The statistical parameters including coefficient of determination, NSE (Nash and Sutcliffe Efficiency), NSEln (modified NSE), and PBIAS (Percent BIAS) were utilized to evaluate the ANN models. From the result, applications of Hyperbolic tangent function and ELU function to the hidden layer and Identity function to the output layer show competent performance rather than other functions which demonstrated the function selection in the ANN structure can affect the performance of ANN.

• International journal of advanced smart convergence
• /
• 제6권4호
• /
• pp.73-79
• /
• 2017

The Convolutional Neural Network (CNN) has shown an excellent performance in computer vision task. Applications of CNN include image classification, object detection in images, autonomous driving, etc. This paper will evaluate the performance of CNN model with ReLU and SELU as activation function. The evaluation will be performed on four different choices of hyperparameter which are initialization method, network configuration, optimization technique, and regularization. We did experiment on each choice of hyperparameter and show how it influences the network convergence and test accuracy. In this experiment, we also discover performance improvement when using SELU as activation function over ReLU.

• 대한전기학회논문지
• /
• 제41권3호
• /
• pp.274-280
• /
• 1992

In this paper, an Improved Error Back Propagation Algorithm is introduced, which avoids Network Paralysis, one of the problems of the Error Backpropagation learning rule. For this purpose, we analyzed the reason for Network Paralysis and modified the Activation Function Derivative of the standard Error Backpropagation Algorithm which is regarded as the cause of the phenomenon. The characteristics of the modified Activation Function Derivative is analyzed. The performance of the modified Error Backpropagation Algorithm is shown to be better than that of the standard Error Back Propagation algorithm by various experiments.

• 한국컴퓨터정보학회논문지
• /
• 제11권3호
• /
• pp.107-115
• /
• 2006

본 논문에서는 캐스케이드 코릴레이션 학습 알고리즘을 위한 새로운 클래스의 활성화 함수를 소개한다. 이 함수는 코사인으로 모듈화된 가우스 함수로서 편의상 이 활성화 함수를 코스가우스(CosGauss) 함수라고 칭하기로 한다. 이 함수는 기존의 시그모이드 함수(sigmoidal function), 하이퍼볼릭탄젠트 함수(hyperbolic tangent function), 가우스 함수(gaussian function)에 비해서 등성이(ridge)를 더 많이 얻을 수 있다. 이러한 등성이들로 인하여 빠른 속도로 수렴하고 패턴인식 속도를 향상 시켜서 학습 능력을 향상시킬 수 있다. 캐스케이드 코릴레이션 네트워크에 이 활성화 함수를 사용하여 중요한 기준 문제(benchmark problem)의 하나인 이중나선 문제(two spirals problem)에 대하여 실험하여 다른 활성화 함수들과 결과 값을 비교하였다.

• 한국산학기술학회논문지
• /
• 제15권3호
• /
• pp.1724-1733
• /
• 2014

본 논문에서는 비모노톤함수(non-monotone function)인 CosExp(cosine-modulated symmetric Exponential function) 함수와 모노톤함수(monotone function)인 시그모이드 함수를 캐스케이드 코릴레이션 알고리즘(Cascade Correlation algorithm)의 학습에 병행해서 사용하여 이중나선문제(two spirals problem)의 패턴인식에 어떠한 영향이 있는지 분석하고 이어서 알고리즘의 최적화를 시도한다. 첫 번째 실험에서는 알고리즘의 후보뉴런에 CosExp 함수를 그리고 출력뉴런에는 시그모이드 함수를 사용하여 나온 인식된 패턴을 분석한다. 두 번째 실험에서는 반대로 CosExp 함수를 출력뉴런에서 사용하고 시그모이드 함수를 후보뉴런에 사용하여 실험하고 결과를 분석한다. 세 번째 실험에서는 후보뉴런을 위한 8개의 풀을 구성하여 변형된 다양한 시그모이드 활성화 함수(sigmoidal activation function)를 사용하고 출력뉴런에는 CosExp함수를 사용하여 얻게 된 입력공간의 인식된 패턴을 분석한다. 네 번째 실험에서는 시그모이드 함수의 변위를 결정하는 세 개의 파라미터 값을 유전자 알고리즘을 이용하여 얻는다. 이 파라미터 값들이 적용된 시그모이드 함수들은 후보뉴런의 활성화를 위해서 사용되고 출력뉴런에는 CosExp 함수를 사용하여 실험한 최적화 된 결과를 분석한다. 이러한 알고리즘의 성능평가를 위하여 각 학습단계 마다 입력패턴공간에서 인식된 이중나선의 형태를 그래픽으로 보여준다. 최적화 과정에서 은닉뉴런(hidden neuron)의 숫자가 28에서 15로 그리고 최종적으로 12개로 줄어서 학습 알고리즘이 최적화되었음을 확인하였다.

• 제어로봇시스템학회논문지
• /
• 제3권6호
• /
• pp.594-601
• /
• 1997

A new algorithm is proposed for training the multilayer perceptrion(MLP) in pattern classification problems to accelerate the learning speed. It is shown that the sigmoid activation function of the output node can have deterimental effect on the performance of learning. To overcome this detrimental effect and to use the information fully in supervised learning, an objective function for binary modes is proposed. This objective function is composed with two new output activation functions which are selectively used depending on desired values of training patterns. The effect of the objective function is analyzed and a training algorithm is proposed based on this. Its performance is tested in several examples. Simulation results show that the performance of the proposed method is better than that of the conventional error back propagation (EBP) method.

• 대한물리의학회지
• /
• 제16권1호
• /
• pp.111-121
• /
• 2021

PURPOSE: This study examined the effects of core stability exercise on the strength, activation of the trunk muscle, and pulmonary function in a Guillain-Barre syndrome (GBS) patient. METHODS: A 38-year-old male with GBS was enrolled in the study. A core stability exercise program was implemented for four weeks with a duration of 30 min/day and a frequency of three days/week. The program consisted of abdominal crunch, Swiss ball crunch, bicycle crunch, medicine ball sit-up with a toss, medicine ball rotational chest pass, raised upper body and lower body, and dead bug. Measurements of the strength of the trunk muscle (trunk flexion and hip flexion), activation of trunk muscles (rectus femoris; RA, external oblique abdominal; EOA, internal oblique abdominal; IOA, erector spinae; ES), and pulmonary function (forced expiratory capacity; FVC, forced expiratory volume at one second; FEV1) were taken before and after four weeks of core stability exercise. RESULTS: The strength of trunk muscles increased in the trunk and hip flexion after four weeks of core stability exercise, respectively, compared to the baseline levels. Activation of the trunk muscles increased in RA, EOA, and IOA after four weeks of core stability exercise compared to baseline levels, but decreased in ES after four weeks of core stability exercise compared to the baseline levels. The pulmonary function increased in FVC and FEV1 after four weeks of core stability exercise compared to the baseline levels. CONCLUSION: These results suggest that core stability exercise improves strength, Activation of the trunk muscle, And pulmonary function in patients with GBS.

• 호남수학학술지
• /
• 제27권2호
• /
• pp.225-232
• /
• 2005

We obtain the approximation order to a smooth function on a compact subset of $\mathbb{R}$ by generalized translation networks. In our study, the activation function is infinitely many times continuously differentiable function but it does not have special properties around ${\infty}$ and $-{\infty}$ like a sigmoidal activation function. Using the Jackson's Theorem, we get the approximation order. Especially, we obtain the approximation order by a neural network with a fixed threshold.

• 제어로봇시스템학회:학술대회논문집
• /
• 제어로봇시스템학회 2000년도 제15차 학술회의논문집
• /
• pp.20-20
• /
• 2000

Back-propagation neural network (BPNN) is the most prevalently used paradigm in modeling semiconductor manufacturing processes, which as a neuron activation function typically employs a bipolar or unipolar sigmoid function in either hidden and output layers. In this study, applicability of another linear function as a neuron activation function is investigated. The linear function was operated in combination with other sigmoid functions. Comparison revealed that a particular combination, the bipolar sigmoid function in hidden layer and the linear function in output layer, is found to be the best combination that yields the highest prediction accuracy. For BPNN with this combination, predictive performance once again optimized by incrementally adjusting the gradients respective to each function. A total of 121 combinations of gradients were examined and out of them one optimal set was determined. Predictive performance of the corresponding model were compared to non-optimized, revealing that optimized models are more accurate over non-optimized counterparts by an improvement of more than 30%. This demonstrates that the proposed gradient-optimized teaming for BPNN with a linear function in output layer is an effective means to construct plasma models. The plasma modeled is a hemispherical inductively coupled plasma, which was characterized by a 24 full factorial design. To validate models, another eight experiments were conducted. process variables that were varied in the design include source polver, pressure, position of chuck holder and chroline flow rate. Plasma attributes measured using Langmuir probe are electron density, electron temperature, and plasma potential.