• Convergence Security Journal
• /
• v.23 no.5
• /
• pp.57-64
• /
• 2023

In this study, we analyzed how to steal the target model without training data. Input data is generated using the generative model, and a similar model is created by defining a loss function so that the predicted values of the target model and the similar model are close to each other. At this time, the target model has a process of learning so that the similar model is similar to it by gradient descent using the logit (logic) value of each class for the input data. The tensorflow machine learning library was used as an experimental environment, and CIFAR10 and SVHN were used as datasets. A similar model was created using the ResNet model as a target model. As a result of the experiment, it was found that the model stealing method generated a similar model with an accuracy of 86.18% for CIFAR10 and 96.02% for SVHN, producing similar predicted values to the target model. In addition, considerations on the model stealing method, military use, and limitations were also analyzed.

• Korean Journal of Biological Psychiatry
• /
• v.27 no.1
• /
• pp.18-26
• /
• 2020

Objectives The aim was to find effective vectorization and classification models to predict a psychiatric diagnosis from text-based medical records. Methods Electronic medical records (n = 494) of present illness were collected retrospectively in inpatient admission notes with three diagnoses of major depressive disorder, type 1 bipolar disorder, and schizophrenia. Data were split into 400 training data and 94 independent validation data. Data were vectorized by two different models such as term frequency-inverse document frequency (TF-IDF) and Doc2vec. Machine learning models for classification including stochastic gradient descent, logistic regression, support vector classification, and deep learning (DL) were applied to predict three psychiatric diagnoses. Five-fold cross-validation was used to find an effective model. Metrics such as accuracy, precision, recall, and F1-score were measured for comparison between the models. Results Five-fold cross-validation in training data showed DL model with Doc2vec was the most effective model to predict the diagnosis (accuracy = 0.87, F1-score = 0.87). However, these metrics have been reduced in independent test data set with final working DL models (accuracy = 0.79, F1-score = 0.79), while the model of logistic regression and support vector machine with Doc2vec showed slightly better performance (accuracy = 0.80, F1-score = 0.80) than the DL models with Doc2vec and others with TF-IDF. Conclusions The current results suggest that the vectorization may have more impact on the performance of classification than the machine learning model. However, data set had a number of limitations including small sample size, imbalance among the category, and its generalizability. With this regard, the need for research with multi-sites and large samples is suggested to improve the machine learning models.

• Journal of KIISE:Software and Applications
• /
• v.26 no.11
• /
• pp.1261-1269
• /
• 1999

다층 퍼셉트론은 다양한 응용 분야에 성공적으로 적용되고 있는 대표적인 신경회로망 모델이다. 그러나 다층 퍼셉트론의 학습에서 나타나는 플라토에 기인한 느린 학습 속도와 지역 극소는 실제 응용문제에 적용함에 있어서 가장 큰 문제로 지적되어왔다. 이 문제를 해결하기 위해 여러 가지 다양한 학습알고리즘들이 개발되어 왔으나, 계산의 비효율성으로 인해 실제 문제에는 적용하기 힘든 예가 많은 등, 현재까지 만족할 만한 해결책은 제시되지 못하고 있다. 본 논문에서는 다층퍼셉트론의 베이시스 함수로 사용되는 시그모이드 함수를 보다 일반화된 형태로 정의하여 사용함으로써 학습에 있어서의 플라토를 완화하고, 지역극소에 빠지는 것을 줄이는 접근방법을 소개한다. 본 방법은 기존의 변형된 가중치 수정식을 사용한 학습 속도 향상의 방법들과는 다른 접근 방법을 택함으로써 기존의 방법들과 함께 사용하는 것이 가능하다는 특징을 갖고 있다. 제안하는 방법의 성능을 확인하기 위하여 간단한 패턴 인식 문제들에의 적용 실험 및 기존의 학습 속도 향상 방법을 함께 사용하여 시계열 예측 문제에 적용한 실험을 수행하였고, 그 결과로부터 제안안 방법의 효율성을 확인할 수 있었다. Abstract A multilayer perceptron is the most well-known neural network model which has been successfully applied to various fields of application. Its slow learning caused by plateau and local minima of gradient descent learning, however, have been pointed as the biggest problems in its practical use. To solve such a problem, a number of researches on learning algorithms have been conducted, but it can be said that none of satisfying solutions have been presented so far because the problems such as computational inefficiency have still been existed in these algorithms. In this paper, we propose a new learning approach to minimize the effect of plateau and reduce the possibility of getting trapped in local minima by generalizing the sigmoidal function which is used as the basis function of a multilayer perceptron. Adapting a new approach that differs from the conventional methods with revised updating equation, the proposed method can be used together with the existing methods to improve the learning performance. We conducted some experiments to test the proposed method on simple problems of pattern recognition and a problem of time series prediction, compared our results with the results of the existing methods, and confirmed that the proposed method is efficient enough to apply to the real problems.

• Journal of The Korean Society of Agricultural Engineers
• /
• v.59 no.4
• /
• pp.97-107
• /
• 2017

This study is to determine the coefficients of regression equations and to select the optimal regression equation in the LOADEST model after classifying the whole study period into 5 flow conditions for 16 watersheds located in the Nakdonggang waterbody. The optimized coefficients of regression equations were derived using the gradient descent method as a learning method in Tensorflow which is the engine of machine-learning method. In South Korea, the variability of streamflow is relatively high, and rainfall is concentrated in summer that can significantly affect the characteristic analysis of pollutant loads. Thus, unlike the previous application of the LOADEST model (adjusting whole study period), the study period was classified into 5 flow conditions to estimate the optimized coefficients and regression equations in the LOADEST model. As shown in the results, the equation #9 which has 7 coefficients related to flow and seasonal characteristics was selected for each flow condition in the study watersheds. When compared the simulated load (SS) to observed load, the simulation showed a similar pattern to the observation for the high flow condition due to the flow parameters related to precipitation directly. On the other hand, although the simulated load showed a similar pattern to observation in several watersheds, most of study watersheds showed large differences for the low flow conditions. This is because the pollutant load during low flow conditions might be significantly affected by baseflow or point-source pollutant load. Thus, based on the results of this study, it can be found that to estimate the continuous pollutant load properly the regression equations need to be determined with proper coefficients based on various flow conditions in watersheds. Furthermore, the machine-learning method can be useful to estimate the coefficients of regression equations in the LOADEST model.

• Journal of the Korean Data and Information Science Society
• /
• v.8 no.2
• /
• pp.163-171
• /
• 1997

Function approximation from a set of input-output pairs has numerous applications in scientific and engineering areas. Multilayer feedforward neural networks have been proposed as a good approximator of nonlinear function. The back propagation(BP) algorithm allows multilayer feedforward neural networks to learn input-output mappings from training samples. It iteratively adjusts the network parameters(weights) to minimize the sum of squared approximation errors using a gradient descent technique. However, the mapping acquired through the BP algorithm may be corrupt when errorneous training data we employed. When errorneous traning data are employed, the learned mapping can oscillate badly between data points. In this paper we propose a robust BP learning algorithm that is resistant to the errorneous data and is capable of rejecting gross errors during the approximation process, that is stable under small noise perturbation and robust against gross errors.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.15 no.4
• /
• pp.457-466
• /
• 2005

In this paper, we propose a wavelet neural network(WNN) based predictive control method for path tracking of mobile robots with multi-input and multi-output. In our control method, we use a WNN as a state predictor which combines the capability of artificial neural networks in learning processes and the capability of wavelet decomposition. A WNN predictor is tuned to minimize errors between the WNN outputs and the states of mobile robot using the gradient descent rule. And control signals, linear velocity and angular velocity, are calculated to minimize the predefined cost function using errors between the reference states and the predicted states. Through a computer simulation for the tracking performance according to varied track, we demonstrate the efficiency and the feasibility of our predictive control system.

• The Transactions of The Korean Institute of Electrical Engineers
• /
• v.61 no.5
• /
• pp.744-752
• /
• 2012

In this study, the Polynomial-based Radial Basis Function Neural Networks is proposed as an one of the recognition part of overall face recognition system that consists of two parts such as the preprocessing part and recognition part. The design methodology and procedure of the proposed pRBFNNs are presented to obtain the solution to high-dimensional pattern recognition problems. In data preprocessing part, Principal Component Analysis(PCA) which is generally used in face recognition, which is useful to express some classes using reduction, since it is effective to maintain the rate of recognition and to reduce the amount of data at the same time. However, because of there of the whole face image, it can not guarantee the detection rate about the change of viewpoint and whole image. Thus, to compensate for the defects, Linear Discriminant Analysis(LDA) is used to enhance the separation of different classes. In this paper, we combine the PCA&LDA algorithm and design the optimized pRBFNNs for recognition module. The proposed pRBFNNs architecture consists of three functional modules such as the condition part, the conclusion part, and the inference part as fuzzy rules formed in 'If-then' format. In the condition part of fuzzy rules, input space is partitioned with Fuzzy C-Means clustering. In the conclusion part of rules, the connection weight of pRBFNNs is represented as two kinds of polynomials such as constant, and linear. The coefficients of connection weight identified with back-propagation using gradient descent method. The output of the pRBFNNs model is obtained by fuzzy inference method in the inference part of fuzzy rules. The essential design parameters (including learning rate, momentum coefficient and fuzzification coefficient) of the networks are optimized by means of Differential Evolution. The proposed pRBFNNs are applied to face image(ex Yale, AT&T) datasets and then demonstrated from the viewpoint of the output performance and recognition rate.

• International Journal of Computer Science & Network Security
• /
• v.22 no.12
• /
• pp.185-196
• /
• 2022

The Internet of Things (IoT) has become more and more widespread in recent years, thus attackers are placing greater emphasis on IoT environments. The IoT connects a large number of smart devices via wired and wireless networks that incorporate sensors or actuators in order to produce and share meaningful information. Attackers employed IoT devices as bots to assault the target server; however, because of their resource limitations, these devices are easily infected with IoT malware. The Distributed Denial of Service (DDoS) is one of the many security problems that might arise in an IoT context. DDOS attempt involves flooding a target server with irrelevant requests in an effort to disrupt it fully or partially. This worst practice blocks the legitimate user requests from being processed. We explored an intelligent intrusion detection system (IIDS) using a particular sort of machine learning, such as Artificial Neural Networks, (ANN) in order to handle and mitigate this type of cyber-attacks. In this research paper Feed-Forward Neural Network (FNN) is tested for detecting the DDOS attacks using a modified version of the KDD Cup 99 dataset. The aim of this paper is to determine the performance of the most effective and efficient Back-propagation algorithms among several algorithms and check the potential capability of ANN- based network model as a classifier to counteract the cyber-attacks in IoT environments. We have found that except Gradient Descent with Momentum Algorithm, the success rate obtained by the other three optimized and effective Back- Propagation algorithms is above 99.00%. The experimental findings showed that the accuracy rate of the proposed method using ANN is satisfactory.

• International Journal of Computer Science & Network Security
• /
• v.24 no.2
• /
• pp.101-112
• /
• 2024

Telugu language is considered as fourth most used language in India especially in the regions of Andhra Pradesh, Telangana, Karnataka etc. In international recognized countries also, Telugu is widely growing spoken language. This language comprises of different dependent and independent vowels, consonants and digits. In this aspect, the enhancement of Telugu Handwritten Character Recognition (HCR) has not been propagated. HCR is a neural network technique of converting a documented image to edited text one which can be used for many other applications. This reduces time and effort without starting over from the beginning every time. In this work, a Unicode based Handwritten Character Recognition(U-HCR) is developed for translating the handwritten Telugu characters into English language. With the use of Centre of Gravity (CG) in our model we can easily divide a compound character into individual character with the help of Unicode values. For training this model, we have used both online and offline Telugu character datasets. To extract the features in the scanned image we used convolutional neural network along with Machine Learning classifiers like Random Forest and Support Vector Machine. Stochastic Gradient Descent (SGD), Root Mean Square Propagation (RMS-P) and Adaptative Moment Estimation (ADAM)optimizers are used in this work to enhance the performance of U-HCR and to reduce the loss function value. This loss value reduction can be possible with optimizers by using CNN. In both online and offline datasets, proposed model showed promising results by maintaining the accuracies with 90.28% for SGD, 96.97% for RMS-P and 93.57% for ADAM respectively.

• Journal of Intelligence and Information Systems
• /
• v.10 no.1
• /
• pp.109-123
• /
• 2004

Corporate financial distress and bankruptcy prediction is one of the major application areas of artificial neural networks (ANNs) in finance and management. ANNs have showed high prediction performance in this area, but sometimes are confronted with inconsistent and unpredictable performance for noisy data. In addition, it may not be possible to train ANN or the training task cannot be effectively carried out without data reduction when the amount of data is so large because training the large data set needs much processing time and additional costs of collecting data. Instance selection is one of popular methods for dimensionality reduction and is directly related to data reduction. Although some researchers have addressed the need for instance selection in instance-based learning algorithms, there is little research on instance selection for ANN. This study proposes a genetic algorithm (GA) approach to instance selection in ANN for bankruptcy prediction. In this study, we use ANN supported by the GA to optimize the connection weights between layers and select relevant instances. It is expected that the globally evolved weights mitigate the well-known limitations of gradient descent algorithm of backpropagation algorithm. In addition, genetically selected instances will shorten the learning time and enhance prediction performance. This study will compare the proposed model with other major data mining techniques. Experimental results show that the GA approach is a promising method for instance selection in ANN.