• 한국지능시스템학회논문지
• /
• 제27권1호
• /
• pp.22-28
• /
• 2017

Convolutional Neural Network(CNN)는 특징 추출과 분류의 두 단계로 나눌 수 있다. 그 중 특징 추출 단계의 커널의 크기, 채널의 수, stride 등의 hyperparameter는 CNN의 구조를 결정할 뿐만 아니라 특징을 추출하는 데에도 영향을 주기 때문에 CNN의 전체적인 성능에도 영향을 준다. 본 논문에서는 Parameter-Setting-Free Harmony Search(PSF-HS) 알고리즘을 이용하여 CNN의 특징 추출 단계에서의 hyperparameter를 최적화 하는 방법을 제안하였다. CNN의 전체 구조를 설정한 뒤 hyperparameter를 변수로 설정하였고 PSF-HS 알고리즘을 적용하여 hyperparameter를 최적화 하였다. 시뮬레이션은 MATLAB을 이용하여 진행하였고 CNN은 mnist 데이터를 이용하여 학습과 테스트를 했다. 총 500번 동안 변수를 업데이트했고 제안하는 방법을 이용하여 구한 CNN 구조 중 가장 높은 정확도를 가지는 구조는 99.28%의 정확도로 mnist 데이터를 분류하는 것을 확인할 수 있었다.

• International Journal of Advanced Culture Technology
• /
• 제7권4호
• /
• pp.56-62
• /
• 2019

Recently, research using artificial neural networks has further expanded the field of neural network optimization and automatic structuring from improving inference accuracy. The performance of the machine learning algorithm depends on how the hyperparameters are configured. Open-source hyperparameter optimization software can be an important step forward in improving the performance of machine learning algorithms. In this paper, we review open-source hyperparameter optimization softwares.

• Geomechanics and Engineering
• /
• 제37권3호
• /
• pp.263-277
• /
• 2024

Artificial Neural Networks (ANNs) are artificial learning algorithms that provide successful results in solving many machine learning problems such as classification, prediction, object detection, object segmentation, image and video classification. There is an increasing number of studies that use ANNs as a prediction tool in soil classification. The aim of this research was to understand the role of hyperparameter optimization in enhancing the accuracy of ANNs for soil type classification. The research results has shown that the hyperparameter optimization and hyperparamter optimized ANNs can be utilized as an efficient mechanism for increasing the estimation accuracy for this problem. It is observed that the developed hyperparameter tool (HyperNetExplorer) that is utilizing the Covariance Matrix Adaptation Evolution Strategy (CMAES), Genetic Algorithm (GA) and Jaya Algorithm (JA) optimization techniques can be successfully used for the discovery of hyperparameter optimized ANNs, which can accomplish soil classification with 100% accuracy.

• 상하수도학회지
• /
• 제35권4호
• /
• pp.259-275
• /
• 2021

This study explored the usefulness and implications of the Bayesian hyperparameter optimization in developing species distribution models (SDMs). A variety of machine learning (ML) algorithms, namely, support vector machine (SVM), random forest (RF), boosted regression tree (BRT), XGBoost (XGB), and Multilayer perceptron (MLP) were used for predicting the occurrence of four benthic macroinvertebrate species. The Bayesian optimization method successfully tuned model hyperparameters, with all ML models resulting an area under the curve (AUC) > 0.7. Also, hyperparameter search ranges that generally clustered around the optimal values suggest the efficiency of the Bayesian optimization in finding optimal sets of hyperparameters. Tree based ensemble algorithms (BRT, RF, and XGB) tended to show higher performances than SVM and MLP. Important hyperparameters and optimal values differed by species and ML model, indicating the necessity of hyperparameter tuning for improving individual model performances. The optimization results demonstrate that for all macroinvertebrate species SVM and RF required fewer numbers of trials until obtaining optimal hyperparameter sets, leading to reduced computational cost compared to other ML algorithms. The results of this study suggest that the Bayesian optimization is an efficient method for hyperparameter optimization of machine learning algorithms.

• International journal of advanced smart convergence
• /
• 제6권4호
• /
• pp.73-79
• /
• 2017

The Convolutional Neural Network (CNN) has shown an excellent performance in computer vision task. Applications of CNN include image classification, object detection in images, autonomous driving, etc. This paper will evaluate the performance of CNN model with ReLU and SELU as activation function. The evaluation will be performed on four different choices of hyperparameter which are initialization method, network configuration, optimization technique, and regularization. We did experiment on each choice of hyperparameter and show how it influences the network convergence and test accuracy. In this experiment, we also discover performance improvement when using SELU as activation function over ReLU.

• 융합신호처리학회논문지
• /
• 제21권3호
• /
• pp.148-153
• /
• 2020

합성곱 신경망 모형에서 높은 정확도를 얻기 위해서는 최적의 하이퍼파라미터를 설정하는 작업이 필요하다. 하지만 높은 성능을 낼 수 있는 하이퍼파라미터 값이 정확히 알려진 바가 없으며, 자료마다 최적의 하이퍼파라미터 값이 달라질 수 있기 때문에 매번 실험을 통해서 찾아야만 한다. 또한, 하이퍼파라미터 값들의 범위가 넓고 조합 수가 많기 때문에 시간과 계산량을 줄이기 위해서는 최적값을 찾기 위한 실험 계획을 먼저 한 후에 탐색을 하는 것이 필요하다. 그러나 아직까지 합성곱 신경망 모형에서 하이퍼파라미터 최적화를 위하여 실험계획법을 이용한 연구 결과가 보고되지 않았다. 본 논문에서는 이미지 분류 문제에서 통계방법 중 하나인 실험계획법의 요인배치법을 이용하여 실험 계획을 하고 합성곱 신경망 분석을 한 후에, 높은 성능을 갖는 값을 중심으로 그리드 탐색을 하여 최적의 하이퍼파라미터를 찾는 방법을 제안한다. 실험 계획을 통하여 각 하이퍼파라미터들의 탐색 범위를 줄인 후에 그리드 탐색을 함으로써 효율적으로 연산량을 줄이고 정확도를 높힐 수 있음을 보였다. 또한 실험 결과에서 모형 성능에 가장 큰 영향을 주는 하이퍼파라미터가 학습률이라는 것을 확인할 수 있었다.

• 말소리와 음성과학
• /
• 제13권1호
• /
• pp.45-51
• /
• 2021

End-to-end (E2E) automatic speech recognition (ASR) has achieved promising performance gains with the introduced self-attention network, Transformer. However, due to training time and the number of hyperparameters, finding the optimal hyperparameter set is computationally expensive. This paper investigates the impact of hyperparameters in the Transformer network to answer two questions: which hyperparameter plays a critical role in the task performance and training speed. The Transformer network for training has two encoder and decoder networks combined with Connectionist Temporal Classification (CTC). We have trained the model with Wall Street Journal (WSJ) SI-284 and tested on devl93 and eval92. Seventeen hyperparameters were selected from the ESPnet training configuration, and varying ranges of values were used for experiments. The result shows that "num blocks" and "linear units" hyperparameters in the encoder and decoder networks reduce Word Error Rate (WER) significantly. However, performance gain is more prominent when they are altered in the encoder network. Training duration also linearly increased as "num blocks" and "linear units" hyperparameters' values grow. Based on the experimental results, we collected the optimal values from each hyperparameter and reduced the WER up to 2.9/1.9 from dev93 and eval93 respectively.

• 대한임베디드공학회논문지
• /
• 제18권6호
• /
• pp.319-325
• /
• 2023

In this study, we proposed a method for hyperparameter optimization in the building and training of a deep learning model designed to process point cloud data collected by a millimeter-wave radar system. The primary aim of this study is to facilitate the deployment of a baseline model in resource-constrained IoT devices. We evaluated a RadHAR baseline deep learning model trained on a public dataset composed of point clouds representing five distinct human activities. Additionally, we introduced a coarse-to-fine hyperparameter optimization procedure, showing substantial potential to enhance model efficiency without compromising predictive performance. Experimental results show the feasibility of significantly reducing model size without adversely impacting performance. Specifically, the optimized model demonstrated a 3.3% improvement in classification accuracy despite a 16.8% reduction in number of parameters compared th the baseline model. In conclusion, this research offers valuable insights for the development of deep learning models for resource-constrained IoT devices, underscoring the potential of hyperparameter optimization and model size reduction strategies. This work contributes to enhancing the practicality and usability of deep learning models in real-world environments, where high levels of accuracy and efficiency in data processing and classification tasks are required.

• International Journal of Computer Science & Network Security
• /
• 제24권2호
• /
• pp.196-202
• /
• 2024

Currently, the second most devastating form of cancer in people, particularly in women, is Breast Cancer (BC). In the healthcare industry, Machine Learning (ML) is commonly employed in fatal disease prediction. Due to breast cancer's favorable prognosis at an early stage, a model is created to utilize the Dataset on Wisconsin Diagnostic Breast Cancer (WDBC). Conversely, this model's overarching axiom is to compare the effectiveness of five well-known ML classifiers, including Logistic Regression (LR), Decision Tree (DT), Random Forest (RF), K-Nearest Neighbor (KNN), and Naive Bayes (NB) with the conventional method. To counterbalance the effect with conventional methods, the overarching tactic we utilized was hyperparameter tuning utilizing the grid search method, which improved accuracy, secondary precision, third recall, and finally the F1 score. In this study hyperparameter tuning model, the rate of accuracy increased from 94.15% to 98.83% whereas the accuracy of the conventional method increased from 93.56% to 97.08%. According to this investigation, KNN outperformed all other classifiers in terms of accuracy, achieving a score of 98.83%. In conclusion, our study shows that KNN works well with the hyper-tuning method. These analyses show that this study prediction approach is useful in prognosticating women with breast cancer with a viable performance and more accurate findings when compared to the conventional approach.

• 한국정보과학회:학술대회논문집
• /
• 한국정보과학회 2007년도 가을 학술발표논문집 Vol.34 No.2 (C)
• /
• pp.485-489
• /
• 2007

Gaussian process classifiers (GPCs) are fully statistical kernel classification models which have a latent function with Gaussian process prior Recently, EP approximation method has been proposed to infer the posterior over the latent function. It can have a special hyperparameter which can treat outliers potentially. In this paper, we propose the outlier robust algorithm which alternates EP and the hyperparameter updating until convergence. We also show its usefulness with the simulation results.