1 |
Y. Bengio, I. Goodfellow, and A. Courville, "Deep learning," MIT Press, 2017.
|
2 |
Y. M. Ko and S. W. Ko, "Alleviation of vanishing gradient problem using parametric activation functions," KIPS Transactions on Software and Data Engineering, Vol.10, No.10, pp.407-420, 2021.
DOI
|
3 |
Y. Qin, X. Wang, and J. Zou, "The optimized deep belief networkswith improved logistic Sigmoid units and their application in faultdiagnosis for planetary gearboxes of wind turbines," IEEE Transactions on Industrial Electronics, Vol.66, No.5, pp.3814-3824, 2018.
DOI
|
4 |
X. Wang, Y. Qin, Y. Wang, S. Xiang, and H. Chen, "ReLTanh: An activation function with vanishing gradient resistance for SAE-based DNNs and its application to rotating machinery fault diagnosis," Neurocomputing, Vol.363, pp.88-98, 2019.
DOI
|
5 |
R. Pascanu, T. Mikolov, and Y. Bengio, "Understanding the exploding gradient problem," arXiv:1211.5063, 2012.
|
6 |
N. Y. Kong, Y. M. Ko, and S. W. Ko, "Performance improvement method of convolutional neural network using agile activation function," KIPS Transactions on Software and Data Engineering, Vol.9, No.7, pp.213-220, 2020.
DOI
|
7 |
V. Nair and G. Hinton, "Rectified linear units improve restricted boltzmann machines," ICML, pp.807-814, 2010.
|
8 |
K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal approximators," Neural Networks, Vol.2, Iss.5, pp.359-366, 1989.
DOI
|
9 |
N. Y. Kong and S. W. Ko, "Performance improvement method of deep neural network using parametric activation functions," Journal of the Korea Contents Association, Vol.21, No.3, pp.616-625, 2021.
DOI
|
10 |
M. Roodschild, J. Gotay Sardinas, and A. Will, "A new approach for the vanishing gradient problem on sigmoid activation," Springer Nature, Vol.20, Iss.4, pp.351-360, 2020.
|
11 |
S. Kong and M. Takatsuka, "Hexpo: A vanishing-proof activation function," International Joint Conference on Neural Networks, pp.2562-2567, 2017.
|
12 |
B. Xu, N. Wang, T. Chen, and M. Li, "Empirical evaluation of rectified activations in convolution network," arXiv:1505. 00853, 2015.
|
13 |
R. Pascanu, T. Mikolov, and Y. Bengio, "On the difficulty of training recurrent neural networks," arXiv:1211.5063, 2013.
|
14 |
K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," International Conference on Computer Vision, arXiv:1502.01852, 2015.
|
15 |
D. Clevert, T. Unterthiner, and S. Hochreiter, "Fast and accurate deep network learning by exponential linear units(ELUs)," arXiv:1511.07289, 2016.
|