• Title/Summary/Keyword: Gradient explosion

Search Result 11, Processing Time 0.027 seconds

GRADIENT EXPLOSION FREE ALGORITHM FOR TRAINING RECURRENT NEURAL NETWORKS

  • HONG, SEOYOUNG;JEON, HYERIN;LEE, BYUNGJOON;MIN, CHOHONG
    • Journal of the Korean Society for Industrial and Applied Mathematics
    • /
    • v.24 no.4
    • /
    • pp.331-350
    • /
    • 2020
  • Exploding gradient is a widely known problem in training recurrent neural networks. The explosion problem has often been coped with cutting off the gradient norm by some fixed value. However, this strategy, commonly referred to norm clipping, is an ad hoc approach to attenuate the explosion. In this research, we opt to view the problem from a different perspective, the discrete-time optimal control with infinite horizon for a better understanding of the problem. Through this perspective, we fathom the region at which gradient explosion occurs. Based on the analysis, we introduce a gradient-explosion-free algorithm that keeps the training process away from the region. Numerical tests show that this algorithm is at least three times faster than the clipping strategy.

Certifying the Characteristics of Artificial Explosion Sounds Traveled through Underground Bedrock Medium (지하 암반 매질을 통과한 인공발파음 특성 규명)

  • Yoon, Sang-Hoon;Bae, Myung-Jin
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.33 no.10C
    • /
    • pp.844-850
    • /
    • 2008
  • This paper stated the proposed algorithm to certify the characteristics of artificial explosion sounds traveled through underground bedrock medium. Artificial explosion that travel through underground bedrock had an attenuation within high frequency bands in increase of a distance with multiple transmission paths phenomenon and inhomogeneity of geological status. In this paper, explosion experiment was made in underground tunnel to verify efficiency of proposed algorithm. The could certify the characteristics of artificial explosion sounds as extracted and numerically quantified the characterized parameter with collected sound sample that traveled through underground bedrock channel.

CHEMICAL EVOLUTION OF THE GALAXY: RADIAL PROPERTIES

  • PARK BYEONG-GON;KANG YONG HEE;LEE SEE-WOO
    • Journal of The Korean Astronomical Society
    • /
    • v.29 no.1
    • /
    • pp.63-73
    • /
    • 1996
  • The previous study of chemical evolution of the Galaxy is extended to the radial properties of the Galactic disk. The present model includes radial dependency of the time-dependent bimodal IMF, radial flow of material in the disk, and the change of type I supernova explosion rate with radial distance from the disk center as model parameters and observed gas and stellar density distributions and metallicity abundance gradient as observational constraints. The results of two models in this study explain the observed gas and stellar density distributions well, with the slope of the gas density gradient in the region of 4.5 kpc$Y_1$ and -0.123dex/kpc in model $Y_2$, respectively, which fit well the observed gradient of -0.l1dex/kpc. The abundance gradient reproduced in model $Y_1$ is getting flatter with decreasing radius, while that in model $Y_2$ is getting steeper, which fits better the observed abundance gradient. This result shows the necessity of exponentially increasing type I supernova explosion rate with decreasing radius in order to explain the observed abundance gradient in the disk. The fitness of observed density distribution and star formation rate distribution justifies the reliability of time-dependent bimodal IMF as a compound quantitative chemical evolution model of the Galaxy. The temporal variations of metallicity gradients for carbon, nitrogen and oxygen are also shown.

  • PDF

Analyzing characteristics of Natural Seismic Sounds and Artificial Seismic Sounds by using Spectrum Gradient (스펙트럼 기울기를 이용한 자연지진음과 인공지진음 특성 분석)

  • Yoon, Sang-Hoon;Bae, Myung-Jin
    • Journal of the Institute of Electronics Engineers of Korea SP
    • /
    • v.46 no.1
    • /
    • pp.79-86
    • /
    • 2009
  • This paper proposed an algorithm for extracting spectrum gradient parameter to analyze the characteristics of natural seismic sounds and artificial seismic sounds. The experiment was performed in various area to raise the reliability. The characteristics of natural seismic sounds and artificial seismic sounds were analyzed by extracting gradient indexes of artificial seismic sounds and natural seismic sounds from the data of experiment by using the proposed algorithm. As a result of the experiment and the analysis, gradient indexes of natural seismic sounds were higher than that of artificial seismic sounds because natural seismic sounds had higher attenuation at high-frequency than artificial seismic sounds did and natural seismic sounds were concentrated in low-frequency band.

Heat Dissipation of Sealed LED Light Fixtures Using Pulsating Heat Pipe Technology

  • Kim, Hyung-Tak;Park, Hae-Kyun;Bang, Kwang-Hyun
    • Journal of Advanced Marine Engineering and Technology
    • /
    • v.36 no.1
    • /
    • pp.64-71
    • /
    • 2012
  • An efficient cooling system is an essential part of the electronic packaging such as a high-luminance LED lighting. A special technology, Pulsating Heat Pipe (PHP), can be applied to improve cooling of a sealed, explosion-proof LED light fixture. In this paper, the characteristics of the pulsating heat pipes in the imposed thermal boundary conditions of LED lightings were experimentally investigated and a PHP device that works free of alignment angle was investigated for cooling of explosion-proof LED lights. Five working fluids of ethanol, FC-72, R-123, water, and acetone were chosen for comparison. The experimental pulsating heat pipe was made of copper tubes of internal diameter of 2.1 mm, 26 turns. A variable heat source of electric heater and an array of cooling fins were attached to the pulsating heat pipe. For the alignment of the heating part at bottom, an optimum charging ratio (liquid fluid volume to total volume) was about 50% for most of the fluids and water showed the highest heat transfer performance. For the alignment of the heating part on top, however, only R-123 worked in an un-looped construction. This unique advantage of R-123 is attributed to its high vapor pressure gradient. Applying these findings, a cooling device for an explosion-proof type of LED light rated 30 W was constructed and tested successfully.

Residual Learning Based CNN for Gesture Recognition in Robot Interaction

  • Han, Hua
    • Journal of Information Processing Systems
    • /
    • v.17 no.2
    • /
    • pp.385-398
    • /
    • 2021
  • The complexity of deep learning models affects the real-time performance of gesture recognition, thereby limiting the application of gesture recognition algorithms in actual scenarios. Hence, a residual learning neural network based on a deep convolutional neural network is proposed. First, small convolution kernels are used to extract the local details of gesture images. Subsequently, a shallow residual structure is built to share weights, thereby avoiding gradient disappearance or gradient explosion as the network layer deepens; consequently, the difficulty of model optimisation is simplified. Additional convolutional neural networks are used to accelerate the refinement of deep abstract features based on the spatial importance of the gesture feature distribution. Finally, a fully connected cascade softmax classifier is used to complete the gesture recognition. Compared with the dense connection multiplexing feature information network, the proposed algorithm is optimised in feature multiplexing to avoid performance fluctuations caused by feature redundancy. Experimental results from the ISOGD gesture dataset and Gesture dataset prove that the proposed algorithm affords a fast convergence speed and high accuracy.

A hierarchical fuzzy controller using structured Takagi-Sugeno type fuzzy inference engine

  • Moon G. Joo;Lee, Jin S.
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 1998.10a
    • /
    • pp.179-184
    • /
    • 1998
  • In this paper, a new hierarchical fuzzy inference system (HFIS) using structured Takagi-Sugeno type fuzzy inference units(FIUs) is proposed. The proposed HFIS not only solves the rule explosion problem in conventional HFIS, but also overcomes the readability problem caused by the structure where outputs of previous level FIUs are used as input variables directly. Gradient descent algorithm is used for adaptation of fuzzy rules. The ball and beam control is performed in computer simulation to illustrate the performance of the proposed controller.

  • PDF

DFT Study for Adsorption and Decomposition Mechanism of Trimethylene Oxide on Al(111) Surface

  • Ye, Cai-Chao;Sun, Jie;Zhao, Feng-Qi;Xu, Si-Yu;Ju, Xue-Hai
    • Bulletin of the Korean Chemical Society
    • /
    • v.35 no.7
    • /
    • pp.2013-2018
    • /
    • 2014
  • The adsorption and decomposition of trimethylene oxide ($C_3H_6O$) molecule on the Al(111) surface were investigated by the generalized gradient approximation (GGA) of density functional theory (DFT). The calculations employed a supercell ($6{\times}6{\times}3$) slab model and three-dimensional periodic boundary conditions. The strong attractive forces between $C_3H_6O$ molecule and Al atoms induce the C-O bond breaking of the ring $C_3H_6O$ molecule. Subsequently, the dissociated radical fragments of $C_3H_6O$ molecule oxidize the Al surface. The largest adsorption energy is about -260.0 kJ/mol in V3, V4 and P2, resulting a ring break at the C-O bond. We also investigated the decomposition mechanism of $C_3H_6O$ molecules on the Al(111) surface. The activation energies ($E_a$) for the dissociations V3, V4 and P2 are 133.3, 166.8 and 174.0 kJ/mol, respectively. The hcp site is the most reactive position for $C_3H_6O$ decomposing.

Development of AI Detection Model based on CCTV Image for Underground Utility Tunnel (지하공동구의 CCTV 영상 기반 AI 연기 감지 모델 개발)

  • Kim, Jeongsoo;Park, Sangmi;Hong, Changhee;Park, Seunghwa;Lee, Jaewook
    • Journal of the Society of Disaster Information
    • /
    • v.18 no.2
    • /
    • pp.364-373
    • /
    • 2022
  • Purpose: The purpose of this paper is to develope smoke detection using AI model for detecting the initial fire in underground utility tunnels using CCTV Method: To improve detection performance of smoke which is high irregular, a deep learning model for fire detection was trained to optimize smoke detection. Also, several approaches such as dataset cleansing and gradient exploding release were applied to enhance model, and compared with results of those. Result: Results show the proposed approaches can improve the model performance, and the final model has good prediction capability according to several indexes such as mAP. However, the final model has low false negative but high false positive capacities. Conclusion: The present model can apply to smoke detection in underground utility tunnel, fixing the defect by linking between the model and the utility tunnel control system.

Adaptive Mesh Refinement for Dealing with Shock Wave Analysis (폭발현상 해석을 위한 적응적 요소망 생성)

  • Jun, Yongtae;Lee, Minhyung
    • Korean Journal of Computational Design and Engineering
    • /
    • v.18 no.6
    • /
    • pp.461-469
    • /
    • 2013
  • Computer simulation with FEM is very useful to analyze hypervelocity impact phenomena that are tremendously expensive or otherwise too impractical to analyze experimentally. Shock physics can be efficiently handled by mesh adaptation which allows finite element mesh to be locally optimized to resolve moving shock wave in explosion. In this paper, an adaptive meshing technique based upon quadtree data structure was applied to resolve ballistic impact phenomena. The technique can adaptively refine a mesh in the neighborhood of a shock and coarsen the mesh for the smooth flow behind the shock according to a criterion. The criterion for refinement and coarsening is based upon the standard deviation of the gradient of shock pressure on the associated field. Shock simulation starts with the rough mesh of the pressure field and mesh density is increased locally under the criterion at each time step. The results show that the mesh adaptation enables to minimize the global computation error of FEM and to increase storage and computational saving compared to the fixed resolution of the conventional static mesh approach.