• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.24 no.1
• /
• pp.49-54
• /
• 2024

Data imbalance is a common issue encountered in classification problems, stemming from a significant disparity in the number of samples between classes within the dataset. Such data imbalance typically leads to problems in classification models, including overfitting, underfitting, and misinterpretation of performance metrics. Methods to address this issue include resampling, augmentation, regularization techniques, and adjustment of loss functions. In this paper, we focus on loss function adjustment, particularly comparing the performance of various configurations of loss functions (Cross Entropy, Balanced Cross Entropy, two settings of Focal Loss: 𝛼 = 1 and 𝛼 = Balanced, Asymmetric Loss) on Multi-Class black-box video data with imbalance issues. The comparison is conducted using the I3D, and R3D_18 models.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.7 no.5
• /
• pp.1044-1051
• /
• 2003

This paper surveys the EBP(Error Back Propagation) learning, the Cross Entropy function and the LBL(Layer By Layer) learning, which are used for learning the MLP(Multi Layer Perceptrons). We compare the merits and demerits of each learning method in the handwritten digit recognition. Although the speed of EBP learning is slower than other learning methods in the initial learning process, its generalization capability is better. Also, the speed of Cross Entropy function that makes up for the weak points of EBP learning is faster than that of EBP learning. But its generalization capability is worse because the error signal of the output layer trains the target vector linearly. The speed of LBL learning is the fastest speed among the other learning methods in the initial learning process. However, it can't train for more after a certain time, it has the lowest generalization capability. Therefore, this paper proposes the standard of selecting the learning method when we apply the MLP.

• The Korean Journal of Applied Statistics
• /
• v.3 no.2
• /
• pp.91-105
• /
• 1990

Several practical prior distributions are derived using the maximum entropy principle. Also, an interactive method for estimating a prior distribution which uses the minimum cross-entropy principle is proposed when there are many prior informations. The consistency of the prior distributions obtained by the entropy principles is discussed.

• Proceedings of the Korea Contents Association Conference
• /
• 2014.11a
• /
• pp.39-40
• /
• 2014

신경회로망의 학습을 위하여 크로스 엔트로피 오차함수가 사용되며, 이의 성능향상을 위하여 확장된 크로스 엔트로피 함수도 제시되었다. 크로스 엔트로피 오차함수는 정보이론에서 제시된 상대 엔트로피(relative entropy)에서 유도된 함수이다. 상대 엔트로 피는 두 확률밀도함수의 발산(divergence) 함수이다. 이 논문에서는 상대 엔트로피와 크로스 엔트로피 관계를 파악한 후, 이를 기반으로 확장된 크로스 엔트로피에 상응하는 새로운 엔트로피 발산 함수를 n=2와 4인 경우에 대하여 유도한다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2024.05a
• /
• pp.709-710
• /
• 2024

Super resolution에서 High-frequency Details를 개선하는 것이 최근 문제이다. 기존에는 Super resolution을 Regression task로 접근하므로써 L2 Loss를 사용하여 이미지가 흐릿하게 되었다. 이를 해결하기위해, Classification task로 바꾸므로써 Cross Entropy Loss을 적용하여 Cross-entropy Super-resolution (CS)를 설계한다. CS를 통해 선명도와 Details이 개선되지만, 저주파의 CE Loss 학습으로인한 Black Artifacts가 발생한다. 그래서, L2 Loss는 저주파와 같이 큰 신호에 더 초점을 맞추므로, 성능 개선을 위해 저주파를 L2 Loss에서, 고주파를 CE Loss에서 학습시킨 Frequency-specific Cross-entropy Super-resolution (FCS)을 제안한다. 우리는 왜곡에 강하며 Human의 인식과 유사한 측정지표인 Learned Perceptual Image Patch Similarity (LPIPS)로 평가한다. 실험한 모든 데이터 셋에서 우리의 FCS는 Baseline보다 LPIPS가 약 1.7배 정도 개선되었다.

• Journal of Korea Society of Industrial Information Systems
• /
• v.27 no.4
• /
• pp.37-45
• /
• 2022

In this paper, a multi agent-multi task assignment problem, which is a representative problem of combinatorial optimization, is presented. The objective of the problem is to determine the coordinated agent-task assignment that maximizes the sum of the achievement rates of each task. The achievement rate is represented as a concave down increasing function according to the number of agents assigned to the task. The problem is expressed as an NP-hard problem with a non-linear objective function. In this paper, to solve the assignment problem, we propose a hybrid cross-entropy algorithm as an effective and efficient solution methodology. In fact, the general cross-entropy algorithm might have drawbacks (e.g., slow update of parameters and premature convergence) according to problem situations. Compared to the general cross-entropy algorithm, the proposed method is designed to be less likely to have the two drawbacks. We show that the performances of the proposed methods are better than those of the general cross-entropy algorithm through numerical experiments.

• Journal of the Semiconductor & Display Technology
• /
• v.22 no.2
• /
• pp.40-44
• /
• 2023

The defect detection based on deep learning requires minimal loss and high accuracy to pinpoint product defects. In this paper, we confirm the loss rate of deep learning training based on disc-shaped artificial scaffold images. It is intended to compare the performance of Cross-Entropy functions used in object detection algorithms. The model was constructed using normal, defective artificial scaffold images and category cross entropy and sparse category cross entropy. The data was repeatedly learned five times using each loss function. The average loss rate, average accuracy, final loss rate, and final accuracy according to the loss function were confirmed.

• Journal of the Korean Institute of Telematics and Electronics C
• /
• v.35C no.6
• /
• pp.67-75
• /
• 1998

Although the nCE(n-th order cross-entropy) error function resolves the incorrect saturation problem of conventional EBP(error back-propagation) algorithm, the performance of MLP's (multilayer perceptrons) trained using the nCE function depends heavily on the order of the nCE function. In this paper, we propose an adaptive learning rate to make the performance of MLP's insensitive to the order of the nCE error. Additionally, we propose a limited error signal of output node to prevent unstable learning due to the adaptive learning rate. The effectiveness of the proposed method is demonstrated in simulations of handwritten digit recognition and thyroid diagnosis tasks.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.21 no.2
• /
• pp.189-194
• /
• 2020

This paper analyzes the gradient descent method, which is the one most used for learning neural networks. Learning means updating a parameter so the loss function is at its minimum. The loss function quantifies the difference between actual and predicted values. The gradient descent method uses the slope of the loss function to update the parameter to minimize error, and is currently used in libraries that provide the best deep learning algorithms. However, these algorithms are provided in the form of a black box, making it difficult to identify the advantages and disadvantages of various gradient descent methods. This paper analyzes the characteristics of the stochastic gradient descent method, the momentum method, the AdaGrad method, and the Adadelta method, which are currently used gradient descent methods. The experimental data used a modified National Institute of Standards and Technology (MNIST) data set that is widely used to verify neural networks. The hidden layer consists of two layers: the first with 500 neurons, and the second with 300. The activation function of the output layer is the softmax function, and the rectified linear unit function is used for the remaining input and hidden layers. The loss function uses cross-entropy error.

• Annual Conference on Human and Language Technology
• /
• 2020.10a
• /
• pp.93-97
• /
• 2020

본 연구는 딥 러닝 기반 의존 구문 분석에서, 학습에 적용하는 손실 함수에 따른 성능을 평가하였다. Pointer Network를 이용한 Left-To-Right 모델을 총 세 가지의 손실 함수(Maximize Golden Probability, Cross Entropy, Local Hinge)를 이용하여 학습시켰다. 그 결과 LH 손실 함수로 학습한 모델이 선행 연구와 같이 MGP 손실 함수로 학습한 것에 비해 UAS/LAS가 각각 0.86%p/0.87%p 상승하였으며, 특히 의존 거리가 먼 경우에 대하여 분석 성능이 크게 향상됨을 확인하였다. 딥러닝 의존 구문 분석기를 구현할 때 학습모델과 입력 표상뿐만 아니라 손실 함수 역시 중요하게 고려되어야 함을 보였다.