• KIPS Transactions on Software and Data Engineering
• /
• v.9 no.7
• /
• pp.213-220
• /
• 2020

The convolutional neural network is composed of convolutional layers and fully connected layers. The nonlinear activation function is used in each layer of the convolutional layer and the fully connected layer. The activation function being used in a neural network is a function that simulates the method of transmitting information in a neuron that can transmit a signal and not send a signal if the input signal is above a certain criterion when transmitting a signal between neurons. The conventional activation function does not have a relationship with the loss function, so the process of finding the optimal solution is slow. In order to improve this, an agile activation function that generalizes the activation function is proposed. The agile activation function can improve the performance of the deep neural network in a way that selects the optimal agile parameter through the learning process using the primary differential coefficient of the loss function for the agile parameter in the backpropagation process. Through the MNIST classification problem, we have identified that agile activation functions have superior performance over conventional activation functions.

• KIPS Transactions on Software and Data Engineering
• /
• v.10 no.10
• /
• pp.407-420
• /
• 2021

Deep neural networks are widely used to solve various problems. However, the deep neural network with a deep hidden layer frequently has a vanishing gradient or exploding gradient problem, which is a major obstacle to learning the deep neural network. In this paper, we propose a parametric activation function to alleviate the vanishing gradient problem that can be caused by nonlinear activation function. The proposed parametric activation function can be obtained by applying a parameter that can convert the scale and location of the activation function according to the characteristics of the input data, and the loss function can be minimized without limiting the derivative of the activation function through the backpropagation process. Through the XOR problem with 10 hidden layers and the MNIST classification problem with 8 hidden layers, the performance of the original nonlinear and parametric activation functions was compared, and it was confirmed that the proposed parametric activation function has superior performance in alleviating the vanishing gradient.

• Molecules and Cells
• /
• v.28 no.3
• /
• pp.183-188
• /
• 2009

TSAd/Lad is a T cell adaptor molecule involved in $p56^{lck}$-mediated T cell activation. To investigate the functions of TSAd in T cells, we generated transgenic (TG) mice expressing the SH2 domain of TSAd (TSAd-SH2) under the control of the $p56^{lck}$ proximal promoter. In T cells from TSAd-SH2 TG mice, T cell receptor (TCR)-mediated early signaling events, such as $Ca^{2+}$ flux and ERK activation, were normal; however, late activation events, such as IL-2 production and proliferation, were significantly reduced. Moreover, TCR-induced cell adhesion to extracellular matrix (ECM) proteins and migration through ECM proteins were defective in T cells from TSAd-SH2 TG mice. Furthermore, the contact hypersensitivity (CHS) reaction, an inflammatory response mainly mediated by T helper 1 (Th1) cells, was inhibited in TSAd-SH2 TG mice. Taken together, these results show that TSAd, particularly the SH2 domain of TSAd, is essential for the effector functions of T cells.

• Journal of the Korea Society of Computer and Information
• /
• v.11 no.3
• /
• pp.107-115
• /
• 2006

This paper presents a new class of activation functions for Cascade Correlation learning algorithm, which herein will be called CosGauss function. This function is a cosine modulated gaussian function. In contrast to the sigmoidal, hyperbolic tangent and gaussian functions, more ridges can be obtained by the CosGauss function. Because of the ridges, it is quickly convergent and improves a pattern recognition speed. Consequently it will be able to improve a learning capability. This function was tested with a Cascade Correlation Network on the two spirals problem and results are compared with those obtained with other activation functions.

• Journal of Multimedia Information System
• /
• v.9 no.1
• /
• pp.21-32
• /
• 2022

Early-stage diagnosis of Alzheimer's Disease (AD) from Cognitively Normal (CN) patients is crucial because treatment at an early stage of AD can prevent further progress in the AD's severity in the future. Recently, computer-aided diagnosis using magnetic resonance image (MRI) has shown better performance in the classification of AD. However, these methods use a traditional machine learning algorithm that requires supervision and uses a combination of many complicated processes. In recent research, the performance of deep neural networks has outperformed the traditional machine learning algorithms. The ability to learn from the data and extract features on its own makes the neural networks less prone to errors. In this paper, a dense neural network is designed for binary classification of Alzheimer's disease. To create a classifier with better results, we studied result of different activation functions in the prediction. We obtained results from 5-folds validations with combinations of different activation functions and compared with each other, and the one with the best validation score is used to classify the test data. In this experiment, features used to train the model are obtained from the ADNI database after processing them using FreeSurfer software. For 5-folds validation, two groups: AD and CN are classified. The proposed DNN obtained better accuracy than the traditional machine learning algorithms and the compared previous studies for AD vs. CN, AD vs. Mild Cognitive Impairment (MCI), and MCI vs. CN classifications, respectively. This neural network is robust and better.

• Steel and Composite Structures
• /
• v.44 no.6
• /
• pp.769-788
• /
• 2022

In this paper a mathematical model for the prediction of the fire resistance of slim-floor steel beams based on an Artificial Neural Network modeling procedure is presented. The artificial neural network models are trained and tested using an analytical database compiled for this purpose from analytical results based on FEM. The proposed model was selected as the optimum from a plethora of alternatives, employing different activation functions in the context of Artificial Neural Network technique. The performance of the developed model was compared against analytical results, employing several performance indices. It was found that the proposed model achieves remarkably improved predictions of the fire resistance of slim-floor steel beams. Moreover, based on the optimum developed AN model a closed-form equation for the estimation of fire resistance is derived, which can prove a useful tool for researchers and engineers, while at the same time can effectively support the teaching of this subject at an academic level.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.33 no.5
• /
• pp.761-773
• /
• 2023

This paper proposes a method to increase the power-analysis resistance of the neural network model's feedforward process by replacing the exponential-based activation function, used in the deep-learning field, with an approximated function especially at the multi-layer perceptron model. Due to its nature, the feedforward process of neural networks calculates secret weight and bias, which already trained, so it has risk of exposure of internal information by side-channel attacks. However, various functions are used as the activation function in neural network, so it's difficult to apply conventional side-channel countermeasure techniques, such as masking, to activation function(especially, to exponential-based activation functions). Therefore, this paper shows that even if an exponential-based activation function is replaced with approximated function of simple form, there is no fatal performance degradation of the model, and than suggests a power-analysis resistant feedforward neural network with exponential-based activation function, by masking approximated function and whole network.

• KIPS Transactions on Software and Data Engineering
• /
• v.9 no.10
• /
• pp.303-308
• /
• 2020

The ReLU(Rectified Linear Unit) function has been dominantly used as a standard activation function in most deep artificial neural network models since it was proposed. Later, Leaky ReLU, Swish, and Mish activation functions were presented to replace ReLU, which showed improved performance over existing ReLU function in image classification task. Therefore, we recognized the need to experiment with whether performance improvements could be achieved by replacing the RELU with other activation functions in the super resolution task. In this paper, the performance was compared by changing the activation functions in EDSR model, which showed stable performance in the super resolution task. As a result, in experiments conducted with changing the activation function of EDSR, when the resolution was converted to double, the existing activation function, ReLU, showed similar or higher performance than the other activation functions used in the experiment. When the resolution was converted to four times, Leaky ReLU and Swish function showed slightly improved performance over ReLU. PSNR and SSIM, which can quantitatively evaluate the quality of images, were able to identify average performance improvements of 0.06%, 0.05% when using Leaky ReLU, and average performance improvements of 0.06% and 0.03% when using Swish. When the resolution is converted to eight times, the Mish function shows a slight average performance improvement over the ReLU. Using Mish, PSNR and SSIM were able to identify an average of 0.06% and 0.02% performance improvement over the RELU. In conclusion, Leaky ReLU and Swish showed improved performance compared to ReLU for super resolution that converts resolution four times and Mish showed improved performance compared to ReLU for super resolution that converts resolution eight times. In future study, we should conduct comparative experiments to replace activation functions with Leaky ReLU, Swish and Mish to improve performance in other super resolution models.

• Research in Community and Public Health Nursing
• /
• v.24 no.4
• /
• pp.427-437
• /
• 2013

Purpose: This study was conducted to test effects of a cognition activation program on ADL, cognitive functions, depression, and QOL in the institutionalized elderly. Methods: The subjects were 40 elders who were admitted to a nursing home, with 20 of them in the experimental group and the other 20 in the control group. Data were collected from September 1 to December 17, 2012. The experimental group participated in a 12 week intervention program. ADL, cognitive functions, depression, and QOL were measured before and after the intervention. Data were analyzed by t-test and regression analysis of the SPSS and AMOS programs. Results: ADL (t=2.468, p=.017), depression (t=-2.625, p=012), and QOL (t=2.428, p=.020) were more significantly improved in the experimental group than in the control group, but there were no statistical differences in cognitive function between the two groups. Cognitive functions had positive effects on ADL (CR=-2.263, p=.024) and depression affected QOL (CR=-7.248, p<.001). Conclusion: The intervention program had positive effects on ADL, depression, and QOL except cognition. Cognitive functions of elders dramatically decrease after age 85, and it takes too much time to improve it. Therefore the program for the improvement of cognitive functions should be provided as early as possible.

• 제어로봇시스템학회:학술대회논문집
• /
• 2001.10a
• /
• pp.43.3-43
• /
• 2001

In conventional RBF network, the activation functions of hidden layers generally are symmetric functions like gaussian function. This has been considered to be one of the limiting factors for the network to speed up learning of actuately describing a given function. To avoid this criticism, we propose a pseudo gaussian function (PGF) whose deviation is changed according to the direction of incoming pattern. This property helps to estimate the given function more effectively with a minimal number of centers because of its flexibility of functional representation. A level set method is used to describe the asymmetric shape of deviation of the pseudo gaussian function. To demonstrate the performance of the proposed network ...