• Journal of KIISE:Software and Applications
• /
• v.30 no.7_8
• /
• pp.672-680
• /
• 2003

A shopbot is a software agent whose goal is to maximize buyer´s satisfaction through automatically gathering the price and quality information of goods as well as the services from on-line sellers. In the response to shopbots´ activities, sellers on the Internet need the agents called pricebots that can help them maximize their own profits. In this paper we adopts Q-learning, one of the model-free reinforcement learning methods as a price-setting algorithm of pricebots. A Q-learned agent increases profitability and eliminates the cyclic price wars when compared with the agents using the myoptimal (myopically optimal) pricing strategy Q-teaming needs to select a sequence of state-action fairs for the convergence of Q-teaming. When the uniform random method in selecting state-action pairs is used, the number of accesses to the Q-tables to obtain the optimal Q-values is quite large. Therefore, it is not appropriate for universal on-line learning in a real world environment. This phenomenon occurs because the uniform random selection reflects the uncertainty of exploitation for the optimal policy. In this paper, we propose a Mixed Nonstationary Policy (MNP), which consists of both the auxiliary Markov process and the original Markov process. MNP tries to keep balance of exploration and exploitation in reinforcement learning. Our experiment results show that the Q-learning agent using MNP converges to the optimal Q-values about 2.6 time faster than the uniform random selection on the average.

• Smart Structures and Systems
• /
• v.4 no.1
• /
• pp.1-17
• /
• 2008

This study proposes a conceptual framework of in-situ vibration tests and analyses for quality appraisal of non-slender, cast-in-place piles with irregular cross-section configuration. It evaluates a frequency index from vibration recordings to a series of impulse loadings that is related to total soil-resistance forces around a pile, so as to assess if the pile achieves the design requirement in terms of bearing capacity. In particular, in-situ pile-vibration tests in sequential are carried out, in which dropping a weight from different heights generates series impulse loadings with low-to-high amplitudes. The high-amplitude impulse is designed in way that the load will generate equivalent static load that is equal to or larger than the designed bearing capacity of the pile. This study then uses empirical mode decomposition and Hilbert spectral analysis for processing the nonstationary, short-period recordings, so as to single out with accuracy the frequency index. Comparison of the frequency indices identified from the recordings to the series loadings with the design-based one would tell if the total soil resistance force remains linear or nonlinear and subsequently for the quality appraisal of the pile. As an example, this study investigates six data sets collected from the in-situ tests of two piles in Taipu water pump project, Jiangshu Province of China. It concludes that the two piles have the actual axial load capacity higher than the designed bearing capacity. The true bearing capacity of the piles under investigation can be estimated with accuracy if the amplitude of impact loadings is further increased and the analyses are calibrated with the static testing results.

• Journal of Korean Society of Disaster and Security
• /
• v.13 no.3
• /
• pp.15-28
• /
• 2020

Hydrological data is very important in understanding the hydrological process and identifying its characteristics to protect human life and property from natural disasters. In particular, hydrological analysis are often performed assuming that hydrological data are stationary. However, recently climate change has raised the issue of climate stationary, and it is necessary to analyze the nonstationary of the climate. In this study, a method to analyze the stationarity of hydrological data was examined using the annual precipitation of 37 meteorological stations with long - term record data. Therefore, in this study, the stationary was determined by analyzing the persistence, trend, and stability using annual precipitation. Overall results showed that a trend was observed in 4 out of 37 stations, stable was investigated at 15 stations, and persistence was shown at 4 stations. In the stationary analysis using the annual precipitation data, 25 stations (67% of 37 stations) were nonstationary.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.17 no.7
• /
• pp.930-938
• /
• 2007

Captions which appear in images include information that relates to the images. In order to obtain the information carried by captions, the methods for text extraction from images have been developed. However, most existing methods can be applied to captions with fixed height of stroke's width. We propose a method which can be applied to various caption size. Our method is based on connected components. And then the edge pixels are detected and grouped into connected components. We analyze the properties of connected components and build a neural network which discriminates connected components which include captions from ones which do not. Experimental data is collected from broadcast programs such as news, documentaries, and show programs which include various height caption. Experimental result is evaluated by two criteria : recall and precision. Recall is the ratio of the identified captions in all the captions in images and the precision is the ratio of the captions in the objects identified as captions. The experiment shows that the proposed method can efficiently extract captions various in size.

• Journal of Korean Society of Transportation
• /
• v.21 no.3
• /
• pp.97-106
• /
• 2003

When a route choice is done under uncertainty, a driver has some expectation of traffic conditions that will occur according to the route chosen. This study tries to build a framework in which we can observe the learning behavior of the drivers' expectations of the travel time under nonstationary environment. In order to investigate how drivers have their subjective expectations on traffic conditions in response to public information, a numerical experiment is carried out. We found that rational expectations(RE) formation about the route travel time can be expressed by the adaptive expectation model when the travel time changes in accordance with the nonstationary process which consists of permanent shock and transient shock. Also, we found that the adaptive parameter of the model converges to the fixed value corresponding to the route conditions.

• Journal of Korea Water Resources Association
• /
• v.38 no.8 s.157
• /
• pp.595-604
• /
• 2005

Precipitation simulation for making the data size larger is an important task for hydrologic analysis. The simulation can be divided into two major categories which are the parametric and nonparametric methods. Also, precipitation simulation depends on time intervals such as daily or hourly rainfall simulations. So far, Markov model is the most favored method for daily precipitation simulation. However, most models are consist of state transition probability by using the homogeneous Markov chain model. In order to make a state vector, the small size of data brings difficulties, and also the assumption of homogeneousness among the state vector in a month causes problems. In other words, the process of daily precipitation mechanism is nonstationary. In order to overcome these problems, this paper focused on the nonparametric method by using uni-variate and multi-variate when simulating a precipitation instead of currently used parametric method.

• 제어로봇시스템학회:학술대회논문집
• /
• 1994.10a
• /
• pp.742-745
• /
• 1994

In this paper we show the performance of neural Chandrasekhar filtering which is a special case for the new method of neural filtering using the artificial neural network systems developed recently for the filtering problems of linear and nonlinear, stationary and nonstationary stochastic signals. The neurofilter developed has either the finite impulse response(FIR) structure or the infinite impulse response(IIR) structure. The neurofilter differs from the conventional linear digital FIR and IIR filters because the artificial neural network system used in the neurofilter has nonlinear structure due to the sigmoid function. Numerical studies for the estimation of a second order Butterworth process are performed by changing the structures of the neurofilter in order to evaluate the performance indices under the changes of the output noises or disturbances. In the numerical studies both Chandrasekhar filtering estimates and true signals are used as the training signals for the neurofilter. The results obtained from the studies verified the capabilities which are essentially necessary for on-line filtering of various stochastic signals.

• Proceedings of the KAIS Fall Conference
• /
• 2010.05b
• /
• pp.1180-1180
• /
• 2010

본 연구는 비안정적인 rework 발생 확률 자체가 납기 및 제품 품질에 매우 나쁜 영향을 미치는 복잡한 제조공정을 대상으로 rework 발생 확률의 변화에 따라 작업의 투입정책(dispatching policy)을 동적으로 변화시킬 수 있는 스케줄링 기법을 제안한다. 본 연구에서는 강화학습(reinforcement learning) 기법을 이용하여 시간의 흐름에 따라 변화하는 rework 발생 확률을 기반으로 작업 투입정책의 모수를 동적으로 조정함으로써 효율적인 투입계획을 수립하는 적응형 스케줄링 알고리즘을 제안하고, 다양한 현실적인 시나리오를 개발하여 그 성능을 테스트한다.

• Journal of The Korean Astronomical Society
• /
• v.23 no.1
• /
• pp.63-70
• /
• 1990

The original axisymmetric, stationary electrodynamic model of the central engine in an active galactic nucleus proposed by Macdonald and Thorne consists of a supermassive black hole with magnetic field lines that pass through the region just outside the event horizon of the black hole. Each magnetic field line rotates with a constant angular velocity which will exceed the speed of light at large radii. Even though the field lines are purely mathematical entities this condition sets a stringent physical constraint on the motion of the magnetic field lines and the particles on them. In this paper we will show that we can remove this auxiliary constraint in our model by allowing nonstationary processes. As a result the magnetic field lines can be twisted and wound up in a region lying outside of the quasi-stationary magnetosphere of the black hole. We conclude that astrophysical jets are formed in that region due to the twisted and wound magnetic field lines powered by the Blandford-Znajek process and the other driving forces.

• Industrial Engineering and Management Systems
• /
• v.7 no.1
• /
• pp.9-22
• /
• 2008

As an extension of a previous work by the authors (Song and Esogbue, 2006), a new algorithm for automated modeling of nonstationary seasonal time series is presented in this paper. Issues relative to the methodology for building automatically seasonal time series models and periodic time series models are addressed. This is achieved by inspecting the trend, estimating the seasonality, determining the orders of the model, and estimating the parameters. As in our previous work, the major instruments used in the model identification process are correlograms of the modeling errors while the least square method is used for parameter estimation. We provide numerical illustrations of the performance of the new algorithms with respect to building both seasonal time series and periodic time series models. Additionally, we consider forecasting and exercise the models on some sample time series problems found in the literature as well as real life problems drawn from the retail industry. In each instance, the models are built automatically avoiding the necessity of any human intervention.