• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.38 no.6
• /
• pp.26-37
• /
• 2001

In order to improve the several problems of the general backpropagation, we propose a method using a fuzzy logic system for automatic tuning of the activation function gain in the backpropagation. First, we researched that the changing of the gain of sigmoid function is equivalent to changing the learning rate, the weights, and the biases. The inputs of the fuzzy logic system were the sensitivity of error respect to the last layer and the mean sensitivity of error respect to the hidden layer, and the output was the gain of the sigmoid function. In order to verify the effectiveness of the proposed method, we performed simulations on the parity problem, function approximation, and pattern recognition. The results show that the proposed method has considerably improved the performance compared to the general backpropagation.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2024.05a
• /
• pp.408-411
• /
• 2024

원격 환경에서의 안전한 데이터 처리를 위한 기술 중 동형암호는 암호화된 데이터 간의 연산을 통한 프라이버시 보존형 연산이 가능하여 최근 딥러닝 연산을 동형암호로 수행하고자 하는 연구가 활발히 진행되고 있다. 그러나 동형암호는 신경망에 존재하는 비산술 활성화함수를 직접적으로 연산할 수 없어 다항함수로 대체하여 연산해야만 하는데, 이로 인해 모델의 정확도가 하락하거나 과도한 연산 부하가 발생하는 등의 비효율성 문제가 발생한다. 본 연구에서는 모델 내의 활성화함수를 서로 다르게 근사하는 접근을 순환신경망(Recurrent Neural Network, RNN)에 적용하여 효율적인 동형암호 연산을 수행하는 방법을 제안하고자 한다.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.15 no.3
• /
• pp.1724-1733
• /
• 2014

This paper presents a pattern recognition analysis of two spirals problem and optimization of Cascade Correlation learning algorithm using in combination with a non-monotone function as CosExp(cosine-modulated symmetric exponential function) and a monotone function as sigmoid function. In addition, the algorithm's optimization is attempted. By using genetic algorithms the optimization of the algorithm will attempt. In the first experiment, by using CosExp activation function for candidate neurons of the learning algorithm is analyzed the recognized pattern in input space of the two spirals problem. In the second experiment, CosExp function for output neurons is used. In the third experiment, the sigmoid activation functions with various parameters for candidate neurons in 8 pools and CosExp function for output neurons are used. In the fourth experiment, the parameters are composed of 8 pools and displacement of the sigmoid function to determine the value of the three parameters is obtained using genetic algorithms. The parameter values applied to the sigmoid activation functions for candidate neurons are used. To evaluate the performance of these algorithms, each step of the training input pattern classification shows the shape of the two spirals. In the optimizing process, the number of hidden neurons was reduced from 28 to15, and finally the learning algorithm with 12 hidden neurons was optimized.

• Proceedings of the Korea Society for Industrial Systems Conference
• /
• 2004.06a
• /
• pp.3-7
• /
• 2004

본 연구에서는 학습기능을 갖는 결정론적 볼츠만 머신에 비단조 뉴런을 이용하여 학습 성능을 수치 시뮬레이션을 통하여 분석한다. 먼저 네트워크의 은닉층에 비단조 및 단조뉴런을 이용한 경우에 대하여 각각 활성화 함수로 시그모이드 함수와 end-cut-off 타입의 비단조함수를 사용한 경우에 대하여 성능을 비교한다. 또한, VHDL을 이용해 설계한 DBM 네트워크에 시그모이드 함수와 end-cut-off 타입의 비단조함수를 사용한 경우에 대하여 시뮬레이션을 통해 수치 시뮬레이션과 성능이 같은지 비교하고 그 유용성을 입증한다.

• Review of KIISC
• /
• v.32 no.1
• /
• pp.31-38
• /
• 2022

함수암호(functional encryption)는 프라이버시를 보호하면서 암호화된 데이터에 대한 연산을 수행할 수 있는 진화된 형태의 암호 기술이다. 비밀키를 가진 수신자에게 평문을 전부 제공하는 기존의 암호와 달리, 함수암호는 특정 연산에 대응하는 비밀키를 가진 수신자에게 평문에 대한 연산 결과만을 제공하기 때문에 데이터에 대한 유연한(fine-grained) 접근 제어가 가능하다. 인공지능과 같은 4차 산업혁명 시대의 대표 기술들은 데이터의 활용을 기반으로 하지만 이 과정에서 데이터 노출로 인한 사용자 프라이버시 침해 문제가 발생할 수 있다. 함수암호는 이러한 문제를 해결할 수 있는 기술로써, 프라이버시 보호와 데이터 경제 활성화를 위한 기반 기술로 활용될 수 있다. 본 논문에서는 함수암호 기술에 대한 개념을 설명하고 관련 연구 동향을 소개한다.

• Journal of Intelligence and Information Systems
• /
• v.10 no.2
• /
• pp.53-63
• /
• 2004

Recurrent-Cascade-Correlation(RCC) is a supervised teaming algorithm that automatically determines the size and topology of the network. RCC adds new hidden neurons one by one and creates a multi-layer structure in which each hidden layer has only one neuron. By second order RCC, new hidden neurons are added to only one hidden layer. These created neurons are not connected to each other. We present a generalization of the RCC Architecture by combining the standard RCC Architecture and the second order RCC Architecture. Whenever a hidden neuron has to be added, the new RCC teaming algorithm automatically determines whether the network topology grows vertically or horizontally. This new algorithm using sigmoid, tanh and new activation functions was tested with the morse-benchmark-problem. Therefore we recognized that the number of hidden neurons was decreased by the experiments of the RCC network generalization which used the activation functions.

• Journal of Sensor Science and Technology
• /
• v.8 no.1
• /
• pp.45-52
• /
• 1999

In this raper, an enhanced learning method is proposed for improving the learning speed of the error back propagation learning algorithm. In order to cope with the premature saturation phenomenon at the initial learning stage, a variation scheme of active functions is introduced by using higher order functions, which does not need much increase of computation load. It naturally changes the learning rate of inter-connection weights to a large value as the derivative of sigmoid function abnormally decrease to a small value during the learning epoch. Also, we suggest the hybrid learning method incorporated the proposed method with the momentum training algorithm. Computer simulation results show that the proposed learning algorithm outperforms the conventional methods such as momentum and delta-bar-delta algorithms.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2022.11a
• /
• pp.603-605
• /
• 2022

추천시스템은 개개인의 성향에 따른 맞춤화 추천이 가능하기 때문에 음악, 영상, 뉴스 등 많은 분야에서 관심을 받고 있다. 일반적인 추천시스템 모델은 블랙박스 모델이기 때문에 추천 결과에 따른 원인 도출을 할 수 없다. 하지만 XAI 의 모델은 이러한 블랙박스 모델의 단점을 해결하고자 제안되었다. 그 중 KGAT 는 Attention Score 를 기반으로 추천 결과에 따른 원인을 알 수 있다. 이와 같은 AI, XAI 등의 딥 러닝 모델에서 각각의 활성화 함수는 상황에 따라 상이한 성능을 나타낸다. 이러한 이유로 인해 데이터에 맞는 활성화 함수를 적용해보는 다양한 시도가 필요하다. 따라서 본 논문은 XAI 추천시스템 모델인 KGAT 의 성능 개선을 위해 여러 활성화 함수를 적용해보고, 실험을 통해 수정한 모델의 성능이 개선됨을 보인다.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.11 no.7
• /
• pp.1387-1393
• /
• 2007

Even if the linearly separable patterns can be classified by the conventional single layer perceptron, the non-linear problems such as XOR can not be classified by it. A fuzzy single layer perceptron can solve the conventional XOR problems by applying fuzzy membership functions. However, in the fuzzy single layer perception, there are a couple disadvantages which are a decision boundary is sometimes vibrating and a convergence may be extremely lowered according to the scopes of the initial values and learning rates. In this paper, for these reasons, we proposed an enhanced fuzzy single layer perceptron algorithm that can prevent from vibration the decision boundary by introducing a bias term and can also reduce the learn time by applying the modified delta rule which include the learning rates and the momentum concept and applying the new linear activation function. Consequently, the simulation results of the XOR and pattern classification problems presented that the proposed method provided the shorter learning time and better convergence than the conventional fuzzy single layer perceptron.

• Journal of IKEEE
• /
• v.22 no.1
• /
• pp.197-200
• /
• 2018

In this paper, sigmoid function, which is difficult to implement at hardware level and has a slow speed, is approximated by using PLAN. We use this as an activation function of MLP structure to reduce resource consumption and speed up. In this paper, we show that the proposed method maintains 95% accuracy in $5{\times}5$ size recognition and 1.83 times faster than GPGPU. We have found that even with similar resources as MLPA accelerators, we use more neurons and converge at higher accuracy and higher speed.