• Title/Summary/Keyword: Gradient descent

Search Result 344, Processing Time 0.029 seconds

Development of optimization teaching and learning materials for artificial intelligence mathematics using ChatGPT and Python (ChatGPT와 파이썬을 활용한 <인공지능 수학>의 최적화 교수·학습 자료 개발 연구)

  • Lee, Seunghoon;Ko, Ho Kyoung
    • Communications of Mathematical Education
    • /
    • v.38 no.3
    • /
    • pp.459-486
    • /
    • 2024
  • The purpose of this study is to enhance understanding and utilization of the core mathematical principles of artificial intelligence, and to develop teaching and learning materials that apply algorithmic thinking and integrated methodologies. To achieve this, teaching and learning materials were developed to implement the concept of optimization through Python using ChatGPT, focusing on mean squared error and gradient descent, structured into a total of five sessions. These materials were applied to high school students, and observations of their understanding, learning methods, and attitudes showed positive responses. As a result, the effectiveness of the AI mathematics optimization teaching and learning materials developed in this study and their applicability in educational settings were confirmed.

A Gradient Boosting Method for Graph Neural Networks (그래프 신경망에 대한 그래디언트 부스팅 기법)

  • Jang, Eunjo;Lee, Ki Yong
    • Annual Conference of KIPS
    • /
    • 2022.11a
    • /
    • pp.574-576
    • /
    • 2022
  • 최근 여러 분야에서 그래프 신경망(graph neural network, GNN)이 활발히 연구되고 있다. 하지만 지금까지 대부분의 GNN 연구는 단일 GNN 모델의 성능을 향상하는 데 집중되었다. 본 논문에서는 앙상블(ensemble) 기법의 대표적 기법인 그래디언트 부스팅(gradient boosting)을 이용하여 GNN의 앙상블 모델을 만드는 방법을 제안한다. 제안 방법은 앞서 만들어진 GNN의 오차를 경사 하강법(gradient descent)을 이용하여 감소시키는 방향으로 다음 GNN을 생성한다. 이 과정을 반복하여 GNN의 최종 앙상블 모델을 얻는다. 실험에서 GNN의 대표적인 모델인 그래프 합성곱 신경망(graph convolutional network, GCN)에 제안 방법을 적용하여 앙상블 모델을 생성한 결과, 단일 GCN 모델에 비해 노드 분류 정확도가 11.3%p까지 증가하였음을 확인하였다.

Stochastic Gradient Descent Optimization Model for Demand Response in a Connected Microgrid

  • Sivanantham, Geetha;Gopalakrishnan, Srivatsun
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.1
    • /
    • pp.97-115
    • /
    • 2022
  • Smart power grid is a user friendly system that transforms the traditional electric grid to the one that operates in a co-operative and reliable manner. Demand Response (DR) is one of the important components of the smart grid. The DR programs enable the end user participation by which they can communicate with the electricity service provider and shape their daily energy consumption patterns and reduce their consumption costs. The increasing demands of electricity owing to growing population stresses the need for optimal usage of electricity and also to look out alternative and cheap renewable sources of electricity. The solar and wind energy are the promising sources of alternative energy at present because of renewable nature and low cost implementation. The proposed work models a smart home with renewable energy units. The random nature of the renewable sources like wind and solar energy brings an uncertainty to the model developed. A stochastic dual descent optimization method is used to bring optimality to the developed model. The proposed work is validated using the simulation results. From the results it is concluded that proposed work brings a balanced usage of the grid power and the renewable energy units. The work also optimizes the daily consumption pattern thereby reducing the consumption cost for the end users of electricity.

Digital signal change through artificial intelligence machine learning method comparison and learning (인공지능 기계학습 방법 비교와 학습을 통한 디지털 신호변화)

  • Yi, Dokkyun;Park, Jieun
    • Journal of Digital Convergence
    • /
    • v.17 no.10
    • /
    • pp.251-258
    • /
    • 2019
  • In the future, various products are created in various fields using artificial intelligence. In this age, it is a very important problem to know the operation principle of artificial intelligence learning method and to use it correctly. This paper introduces artificial intelligence learning methods that have been known so far. Learning of artificial intelligence is based on the fixed point iteration method of mathematics. The GD(Gradient Descent) method, which adjusts the convergence speed based on the fixed point iteration method, the Momentum method to summate the amount of gradient, and finally, the Adam method that mixed these methods. This paper describes the advantages and disadvantages of each method. In particularly, the Adam method having adaptivity controls learning ability of machine learning. And we analyze how these methods affect digital signals. The changes in the learning process of digital signals are the basis of accurate application and accurate judgment in the future work and research using artificial intelligence.

A Hybrid Multi-Level Feature Selection Framework for prediction of Chronic Disease

  • G.S. Raghavendra;Shanthi Mahesh;M.V.P. Chandrasekhara Rao
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.12
    • /
    • pp.101-106
    • /
    • 2023
  • Chronic illnesses are among the most common serious problems affecting human health. Early diagnosis of chronic diseases can assist to avoid or mitigate their consequences, potentially decreasing mortality rates. Using machine learning algorithms to identify risk factors is an exciting strategy. The issue with existing feature selection approaches is that each method provides a distinct set of properties that affect model correctness, and present methods cannot perform well on huge multidimensional datasets. We would like to introduce a novel model that contains a feature selection approach that selects optimal characteristics from big multidimensional data sets to provide reliable predictions of chronic illnesses without sacrificing data uniqueness.[1] To ensure the success of our proposed model, we employed balanced classes by employing hybrid balanced class sampling methods on the original dataset, as well as methods for data pre-processing and data transformation, to provide credible data for the training model. We ran and assessed our model on datasets with binary and multivalued classifications. We have used multiple datasets (Parkinson, arrythmia, breast cancer, kidney, diabetes). Suitable features are selected by using the Hybrid feature model consists of Lassocv, decision tree, random forest, gradient boosting,Adaboost, stochastic gradient descent and done voting of attributes which are common output from these methods.Accuracy of original dataset before applying framework is recorded and evaluated against reduced data set of attributes accuracy. The results are shown separately to provide comparisons. Based on the result analysis, we can conclude that our proposed model produced the highest accuracy on multi valued class datasets than on binary class attributes.[1]

Nonlinear Prediction of Time Series Using Multilayer Neural Networks of Hybrid Learning Algorithm (하이브리드 학습알고리즘의 다층신경망을 이용한 시급수의 비선형예측)

  • 조용현;김지영
    • Proceedings of the IEEK Conference
    • /
    • 1998.10a
    • /
    • pp.1281-1284
    • /
    • 1998
  • This paper proposes an efficient time series prediction of the nonlinear dynamical discrete-time systems using multilayer neural networks of a hybrid learning algorithm. The proposed learning algorithm is a hybrid backpropagation algorithm based on the steepest descent for high-speed optimization and the dynamic tunneling for global optimization. The proposed algorithm has been applied to the y00 samples of 700 sequences to predict the next 100 samples. The simulation results shows that the proposed algorithm has better performances of the convergence and the prediction, in comparision with that using backpropagation algorithm based on the gradient descent for multilayer neural network.

  • PDF

Time Series Prediction Using a Multi-layer Neural Network with Low Pass Filter Characteristics (저주파 필터 특성을 갖는 다층 구조 신경망을 이용한 시계열 데이터 예측)

  • Min-Ho Lee
    • Journal of Advanced Marine Engineering and Technology
    • /
    • v.21 no.1
    • /
    • pp.66-70
    • /
    • 1997
  • In this paper a new learning algorithm for curvature smoothing and improved generalization for multi-layer neural networks is proposed. To enhance the generalization ability a constraint term of hidden neuron activations is added to the conventional output error, which gives the curvature smoothing characteristics to multi-layer neural networks. When the total cost consisted of the output error and hidden error is minimized by gradient-descent methods, the additional descent term gives not only the Hebbian learning but also the synaptic weight decay. Therefore it incorporates error back-propagation, Hebbian, and weight decay, and additional computational requirements to the standard error back-propagation is negligible. From the computer simulation of the time series prediction with Santafe competition data it is shown that the proposed learning algorithm gives much better generalization performance.

  • PDF

Design of Equalizer using Fussy Stochastic Gradient Algorithm (퍼지 확률 기울기 알고리즘을 이용한 등화기 설계)

  • Park, Hyoung-Keun;Ra, Yoo-Chan
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.9 no.1
    • /
    • pp.152-159
    • /
    • 2005
  • For high-speed data communication in band-limited channels, main of the bit error are fading and ISI(Inter-Symbol Interference). The common way of dealing with ISI is using equalization in the receiver. In this thesis, channel adaptive equalizer which uses Fuzzy Stochastic Gradient(FSG) and Constant Modulus Algorithm(CMA) is nonlinear equalizer, or Blind equalizer, that works directly on the signals with no training sequences required. This equalizer employs Takagi-Sugeno's fuzzy model that uses the FSG algorithm, to automatically regulate the step size of the descent gradient vector, combining fast convergence rate and low mean square error(MSE), and the CMA which is a special case of Godard's algorithm, to having multiple dispersion constants($R_p$).

Implementation of adaptive filters using fast hadamard transform (고속하다마드 변환을 이용한 적응 필터의 구현)

  • 곽대연;박진배;윤태성
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 1997.10a
    • /
    • pp.1379-1382
    • /
    • 1997
  • We introduce a fast implementation of the adaptive transversal filter which uses least-mean-square(LMS) algorithm. The fast Hadamard transform(FHT) is used for the implementation of the filter. By using the proposed filter we can get the significant time reduction in computatioin over the conventional time domain LMS filter at the cost of a little performance. By computer simulation, we show the comparison of the propsed Hadamard-domain filter and the time domain filter in the view of multiplication time, mean-square error and robustness for noise.

  • PDF

Stable Tracking Control to a Non-linear Process Via Neural Network Model

  • Zhai, Yujia
    • Journal of the Korea Convergence Society
    • /
    • v.5 no.4
    • /
    • pp.163-169
    • /
    • 2014
  • A stable neural network control scheme for unknown non-linear systems is developed in this paper. While the control variable is optimised to minimize the performance index, convergence of the index is guaranteed asymptotically stable by a Lyapnov control law. The optimization is achieved using a gradient descent searching algorithm and is consequently slow. A fast convergence algorithm using an adaptive learning rate is employed to speed up the convergence. Application of the stable control to a single input single output (SISO) non-linear system is simulated. The satisfactory control performance is obtained.