• Title/Summary/Keyword: Layer-by-layer learning

Search Result 650, Processing Time 0.024 seconds

Feasibility of Deep Learning Algorithms for Binary Classification Problems (이진 분류문제에서의 딥러닝 알고리즘의 활용 가능성 평가)

  • Kim, Kitae;Lee, Bomi;Kim, Jong Woo
    • Journal of Intelligence and Information Systems
    • /
    • v.23 no.1
    • /
    • pp.95-108
    • /
    • 2017
  • Recently, AlphaGo which is Bakuk (Go) artificial intelligence program by Google DeepMind, had a huge victory against Lee Sedol. Many people thought that machines would not be able to win a man in Go games because the number of paths to make a one move is more than the number of atoms in the universe unlike chess, but the result was the opposite to what people predicted. After the match, artificial intelligence technology was focused as a core technology of the fourth industrial revolution and attracted attentions from various application domains. Especially, deep learning technique have been attracted as a core artificial intelligence technology used in the AlphaGo algorithm. The deep learning technique is already being applied to many problems. Especially, it shows good performance in image recognition field. In addition, it shows good performance in high dimensional data area such as voice, image and natural language, which was difficult to get good performance using existing machine learning techniques. However, in contrast, it is difficult to find deep leaning researches on traditional business data and structured data analysis. In this study, we tried to find out whether the deep learning techniques have been studied so far can be used not only for the recognition of high dimensional data but also for the binary classification problem of traditional business data analysis such as customer churn analysis, marketing response prediction, and default prediction. And we compare the performance of the deep learning techniques with that of traditional artificial neural network models. The experimental data in the paper is the telemarketing response data of a bank in Portugal. It has input variables such as age, occupation, loan status, and the number of previous telemarketing and has a binary target variable that records whether the customer intends to open an account or not. In this study, to evaluate the possibility of utilization of deep learning algorithms and techniques in binary classification problem, we compared the performance of various models using CNN, LSTM algorithm and dropout, which are widely used algorithms and techniques in deep learning, with that of MLP models which is a traditional artificial neural network model. However, since all the network design alternatives can not be tested due to the nature of the artificial neural network, the experiment was conducted based on restricted settings on the number of hidden layers, the number of neurons in the hidden layer, the number of output data (filters), and the application conditions of the dropout technique. The F1 Score was used to evaluate the performance of models to show how well the models work to classify the interesting class instead of the overall accuracy. The detail methods for applying each deep learning technique in the experiment is as follows. The CNN algorithm is a method that reads adjacent values from a specific value and recognizes the features, but it does not matter how close the distance of each business data field is because each field is usually independent. In this experiment, we set the filter size of the CNN algorithm as the number of fields to learn the whole characteristics of the data at once, and added a hidden layer to make decision based on the additional features. For the model having two LSTM layers, the input direction of the second layer is put in reversed position with first layer in order to reduce the influence from the position of each field. In the case of the dropout technique, we set the neurons to disappear with a probability of 0.5 for each hidden layer. The experimental results show that the predicted model with the highest F1 score was the CNN model using the dropout technique, and the next best model was the MLP model with two hidden layers using the dropout technique. In this study, we were able to get some findings as the experiment had proceeded. First, models using dropout techniques have a slightly more conservative prediction than those without dropout techniques, and it generally shows better performance in classification. Second, CNN models show better classification performance than MLP models. This is interesting because it has shown good performance in binary classification problems which it rarely have been applied to, as well as in the fields where it's effectiveness has been proven. Third, the LSTM algorithm seems to be unsuitable for binary classification problems because the training time is too long compared to the performance improvement. From these results, we can confirm that some of the deep learning algorithms can be applied to solve business binary classification problems.

Physiological Neuro-Fuzzy Learning Algorithm for Face Recognition

  • Kim, Kwang-Baek;Woo, Young-Woon;Park, Hyun-Jung
    • Journal of information and communication convergence engineering
    • /
    • v.5 no.1
    • /
    • pp.50-53
    • /
    • 2007
  • This paper presents face features detection and a new physiological neuro-fuzzy learning method by using two-dimensional variances based on variation of gray level and by learning for a statistical distribution of the detected face features. This paper reports a method to learn by not using partial face image but using global face image. Face detection process of this method is performed by describing differences of variance change between edge region and stationary region by gray-scale variation of global face having featured regions including nose, mouse, and couple of eyes. To process the learning stage, we use the input layer obtained by statistical distribution of the featured regions for performing the new physiological neuro-fuzzy algorithm.

A layered-wise data augmenting algorithm for small sampling data (적은 양의 데이터에 적용 가능한 계층별 데이터 증강 알고리즘)

  • Cho, Hee-chan;Moon, Jong-sub
    • Journal of Internet Computing and Services
    • /
    • v.20 no.6
    • /
    • pp.65-72
    • /
    • 2019
  • Data augmentation is a method that increases the amount of data through various algorithms based on a small amount of sample data. When machine learning and deep learning techniques are used to solve real-world problems, there is often a lack of data sets. The lack of data is at greater risk of underfitting and overfitting, in addition to the poor reflection of the characteristics of the set of data when learning a model. Thus, in this paper, through the layer-wise data augmenting method at each layer of deep neural network, the proposed method produces augmented data that is substantially meaningful and shows that the method presented by the paper through experimentation is effective in the learning of the model by measuring whether the method presented by the paper improves classification accuracy.

Exercise Recommendation System Using Deep Neural Collaborative Filtering (신경망 협업 필터링을 이용한 운동 추천시스템)

  • Jung, Wooyong;Kyeong, Chanuk;Lee, Seongwoo;Kim, Soo-Hyun;Sun, Young-Ghyu;Kim, Jin-Young
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.22 no.6
    • /
    • pp.173-178
    • /
    • 2022
  • Recently, a recommendation system using deep learning in social network services has been actively studied. However, in the case of a recommendation system using deep learning, the cold start problem and the increased learning time due to the complex computation exist as the disadvantage. In this paper, the user-tailored exercise routine recommendation algorithm is proposed using the user's metadata. Metadata (the user's height, weight, sex, etc.) set as the input of the model is applied to the designed model in the proposed algorithms. The exercise recommendation system model proposed in this paper is designed based on the neural collaborative filtering (NCF) algorithm using multi-layer perceptron and matrix factorization algorithm. The learning proceeds with proposed model by receiving user metadata and exercise information. The model where learning is completed provides recommendation score to the user when a specific exercise is set as the input of the model. As a result of the experiment, the proposed exercise recommendation system model showed 10% improvement in recommended performance and 50% reduction in learning time compared to the existing NCF model.

Recognition of hand written Hangul by neural network

  • Song, Jeong-Young;Lee, Hee-Hyol;Choi, Won-Kyu;Akizuki, Kageo
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 1993.10b
    • /
    • pp.76-80
    • /
    • 1993
  • In this paper we discuss optimization of neural network parameters, such as inclination of the sigmoid function, the numbers of the input layer's units and the hidden layer's units, considering application to recognition of hand written Hangul. Hangul characters are composed of vowels and consonants, and basically classified to six patterns by their positions. Using these characteristics of Hangul, the pattern of a given character is determined by its peripheral distribution and the other features. After then, the vowels and the consonants are recognized by the optimized neural network. The constructed recognition system including a neural network is applied to non-learning Hangul written by some Korean people, which are the names randomly taken from Korean spiritual and cultural research institute.

  • PDF

Design of Fuzzy Relation-based Fuzzy Neural Networks with Multi-Output and Its Optimization (다중 출력을 가지는 퍼지 관계 기반 퍼지뉴럴네트워크 설계 및 최적화)

  • Park, Keon-Jun;Kim, Hyun-Ki;Oh, Sung-Kwun
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.58 no.4
    • /
    • pp.832-839
    • /
    • 2009
  • In this paper, we introduce an design of fuzzy relation-based fuzzy neural networks with multi-output. Fuzzy relation-based fuzzy neural networks comprise the network structure generated by dividing the entire input space. The premise part of the fuzzy rules of the network reflects the relation of the division space for the entire input space and the consequent part of the fuzzy rules expresses three types of polynomial functions such as constant, linear, and modified quadratic. For the multi-output structure the neurons in the output layer were connected with connection weights. The learning of fuzzy neural networks is realized by adjusting connections of the neurons both in the consequent part of the fuzzy rules and in the output layer, and it follows a back-propagation algorithm. In addition, in order to optimize the network, the parameters of the network such as apexes of membership functions, learning rate and momentum coefficient are automatically optimized by using real-coded genetic algorithm. Two examples are included to evaluate the performance of the proposed network.

Optimal Structures of a Neural Network Based on OpenCV for a Golf Ball Recognition (골프공 인식을 위한 OpenCV 기반 신경망 최적화 구조)

  • Kim, Kang-Chul
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.10 no.2
    • /
    • pp.267-274
    • /
    • 2015
  • In this paper the optimal structure of a neural network based on OpenCV for a golf ball recognition and the intensity of ROI(Region Of Interest) are calculated. The system is composed of preprocess, image processing and machine learning, and a learning model is obtained by multi-layer perceptron using the inputs of 7 Hu's invariant moments, box ration extracted by vertical and horizontal length or ${\pi}$ calculated by area of ROI. Simulation results show that optimal numbers of hidden layer and the node of neuron are selected to 2 and 9 respectively considering the recognition rate and running time, and optimal intensity of ROI is selected to 200.

Comparison of Off-the-Shelf DCNN Models for Extracting Bark Feature and Tree Species Recognition Using Multi-layer Perceptron (수피 특징 추출을 위한 상용 DCNN 모델의 비교와 다층 퍼셉트론을 이용한 수종 인식)

  • Kim, Min-Ki
    • Journal of Korea Multimedia Society
    • /
    • v.23 no.9
    • /
    • pp.1155-1163
    • /
    • 2020
  • Deep learning approach is emerging as a new way to improve the accuracy of tree species identification using bark image. However, the approach has not been studied enough because it is confronted with the problem of acquiring a large volume of bark image dataset. This study solved this problem by utilizing a pretrained off-the-shelf DCNN model. It compares the discrimination power of bark features extracted by each DCNN model. Then it extracts the features by using a selected DCNN model and feeds them to a multi-layer perceptron (MLP). We found out that the ResNet50 model is effective in extracting bark features and the MLP could be trained well with the features reduced by the principal component analysis. The proposed approach gives accuracy of 99.1% and 98.4% for BarkTex and Trunk12 datasets respectively.

Improving the Water Level Prediction of Multi-Layer Perceptron with a Modified Error Function

  • Oh, Sang-Hoon
    • International Journal of Contents
    • /
    • v.13 no.4
    • /
    • pp.23-28
    • /
    • 2017
  • Of the total economic loss caused by disasters, 40% are due to floods and floods have a severe impact on human health and life. So, it is important to monitor the water level of a river and to issue a flood warning during unfavorable circumstances. In this paper, we propose a modified error function to improve a hydrological modeling using a multi-layer perceptron (MLP) neural network. When MLP's are trained to minimize the conventional mean-squared error function, the prediction performance is poor because MLP's are highly tunned to training data. Our goal is achieved by preventing overspecialization to training data, which is the main reason for performance degradation for rare or test data. Based on the modified error function, an MLP is trained to predict the water level with rainfall data at upper reaches. Through simulations to predict the water level of Nakdong River near a UNESCO World Heritage Site "Hahoe Village," we verified that the prediction performance of MLP with the modified error function is superior to that with the conventional mean-squared error function, especially maximum error of 40.85cm vs. 55.51cm.

Prediction of Wind Power Generation for Calculation of ESS Capacity using Multi-Layer Perceptron (ESS 용량 산정을 위한 다층 퍼셉트론을 이용한 풍력 발전량 예측)

  • Choi, Jeong-Gon;Choi, Hyo-Sang
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.16 no.2
    • /
    • pp.319-328
    • /
    • 2021
  • In this paper, we perform prediction of amount of electric power plant for complex of wind plant using multi-layer perceptron in order to calculate exact calculation of capacity of ESS to maximize profit through generation and to minimize generation cost of wind generation. We acquire wind speed, direction of wind and air density as variables to predict the amount of generation of wind power. Then, we merge and normalize there variables. To train model, we divide merged variables into data as train and test data with ratio of 70% versus 30%. Then we train model by using training data, and we alsouate the prediction performance of model by using test data. Finally, we present the result of prediction in amount of wind power.