• East Asian mathematical journal
• /
• v.10 no.1
• /
• pp.211-218
• /
• 1994

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.3
• /
• pp.837-852
• /
• 2021

Recurrent neural network (RNN) architectures have been used for language modeling (LM) tasks that require learning long-range word or character sequences. However, the RNN architecture is still suffered from unstable gradients on long-range sequences. To address the issue of long-range sequences, an attention mechanism has been used, showing state-of-the-art (SOTA) performance in all LM tasks. A differentiable neural computer (DNC) is a deep learning architecture using an attention mechanism. The DNC architecture is a neural network augmented with a content-addressable external memory. However, in the write operation, some information unrelated to the input word remains in memory. Moreover, DNCs have been found to perform poorly with low numbers of weight parameters. Therefore, we propose a robust memory deallocation method using a limited retention vector. The limited retention vector determines whether the network increases or decreases its usage of information in external memory according to a threshold. We experimentally evaluate the robustness of a DNC implementing the proposed approach according to the size of the controller and external memory on the enwik8 LM task. When we decreased the number of weight parameters by 32.47%, the proposed DNC showed a low bits-per-character (BPC) degradation of 4.30%, demonstrating the effectiveness of our approach in language modeling tasks.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.25 no.4
• /
• pp.501-507
• /
• 2021

Neural networks with differentiable activation functions are differentiable with respect to input variables. We improve the approximation capability of neural networks by using the gradient and Hessian of neural networks to satisfy the differential equations of the problems of interest. We apply differential neural networks to the pricing of financial options, where stochastic differential equations and the Black-Scholes partial differential equation represent the differential relation of price of option and underlying assets, and the first and second derivatives of option price play an important role in financial engineering. The proposed neural network learns - (a) the sample paths of option prices generated by stochastic differential equations and (b) the Black-Scholes equation at each time and asset price. Experimental results show that the proposed method gives accurate option values and the first and second derivatives.

• Bulletin of the Korean Mathematical Society
• /
• v.42 no.3
• /
• pp.535-541
• /
• 2005

In this paper, we define the ap-Denjoy integral and investigate some properties of the ap-Denjoy integral.

• Bulletin of the Korean Mathematical Society
• /
• v.44 no.3
• /
• pp.523-536
• /
• 2007

Sharp error estimates in approximating the Stieltjes integral with bounded integrands and bounded integrators respectively, are given. Applications for three point quadrature rules of n-time differentiable functions are also provided.

• Journal of the Chungcheong Mathematical Society
• /
• v.17 no.1
• /
• pp.103-110
• /
• 2004

In this paper, we define the ap-Denjoy integral and investigate some properties of the ap-Denjoy integral.

• Communications of the Korean Mathematical Society
• /
• v.29 no.2
• /
• pp.345-349
• /
• 2014

We show that $C^1$-generically, a differentiable map is positively measure expansive if and only if it is expanding.

• Bulletin of the Korean Mathematical Society
• /
• v.32 no.1
• /
• pp.1-11
• /
• 1995

Let $p, q \in R^2 and H : R^{2n} \to R^n$ be differentiable. An autonomous Hamiltonian system has the form $$ (0.1) \dot{p} = -\frac{\partial q}{\partial H}(p, q), \dot{q} = \frac{\partial p}{\partial H}(p, q) $$.

• Communications of the Korean Mathematical Society
• /
• v.26 no.1
• /
• pp.125-133
• /
• 2011

A new kind of structure is introduced in an even dimensional differentiable Riemannian manifold and some basic properties of this structure is discussed. Also the existence of such structure is shown with an example.

• Korean Journal of Mathematics
• /
• v.4 no.2
• /
• pp.125-133
• /
• 1996

We show that if X is a reflexive Banach space with a uniformly G$\hat{a}$teaux differentiable norm, then the convergence of semigroups acting on Banach spaces $X_n$ implies the convergence of resolvents of generators of semigroups.