• 품질경영학회지
• /
• 제11권2호
• /
• pp.2-9
• /
• 1983

As will be seen, this paper is tries that the research on the mathematical structure of Markov process and Markovian sequential decision process (the policy improvement iteration method,) moreover, that it analyze the logic and the characteristic of behavior of mathematical model of Markov process. Therefore firstly, it classify, on research of mathematical structure of Markov process, the forward equation and backward equation of Chapman-kolmogorov equation and of kolmogorov differential equation, and then have survey on logic of equation systems or on the question of uniqueness and existence of solution of the equation. Secondly, it classify, at the Markovian sequential decision process, the case of discrete time parameter and the continuous time parameter, and then it explore the logic system of characteristic of the behavior, the value determination operation and the policy improvement routine.

• Communications for Statistical Applications and Methods
• /
• 제21권6호
• /
• pp.513-520
• /
• 2014

We study numerical methods to obtain the stationary probabilities of continuous-time Markov chains whose embedded chains are periodic. The power method is applied to the balance equations of the periodic embedded Markov chains. The power method can have the convergence speed of exponential rate that is ambiguous in its application to original continuous-time Markov chains since the embedded chains are discrete-time processes. An illustrative example is presented to investigate the numerical iteration of this paper. A numerical study shows that a rapid and stable solution for stationary probabilities can be achieved regardless of periodicity and initial conditions.

• 한국수학교육학회지시리즈A:수학교육
• /
• 제25권1호
• /
• pp.1-8
• /
• 1986

It is a common saying that markov-chain is a special case of probability course. That is to say, It means an unchangeable markov-chain process of the transition-probability of discontinuous time. There are two kinds of ways to show transition probability parade matrix theory. The first is the way by arrangement of a rightangled tetragon. The second part is a vertical measurement and direction sing by transition-circle. In this essay, I try to find out existence of procession for transition-probability applied markov-chain. And it is possible for me to know not only, what it is basic on a study of chain but also being applied to abnormal problems following a flow change and statistic facts expecting to use as a model of air expansion in physics.

• 한국시뮬레이션학회논문지
• /
• 제13권3호
• /
• pp.1-10
• /
• 2004

The cell loss probability required in the ATM network is in the range of 10$^{-9}$ ∼10$^{-12}$ . If Monte Carlo simulation is used to analyze the performance of the ATM node, an enormous amount of computer time is required. To obtain large speed-up factors, importance sampling may be used. Since the Markov-modulated processes have been used to model various high-speed network traffic sources, we consider discrete time single server queueing systems with Markov-modulated arrival processes which can be used to model an ATM node. We apply importance sampling based on the Large Deviation Theory for the performance evaluation of, MMBP/D/1/K, ∑MMBP/D/1/K, and two stage tandem queueing networks with Markov-modulated arrival processes and deterministic service times. The simulation results show that the buffer overflow probabilities obtained by the importance sampling are very close to those obtained by the Monte Carlo simulation and the computer time can be reduced drastically.

• 한국경영과학회지
• /
• 제16권1호
• /
• pp.76-88
• /
• 1991

Stochastic convexity (concavity) of a stochastic process is a very useful concept for various stochastic optimization problems. In this study we first establish stochastic convexity of a certain class of Markov additive processes through probabilistic construction based on the sample path approach. A Markov additive process is abtained by integrating a functional of the underlying Markov process with respect to time, and its stochastic convexity can be utilized to provide efficient methods for optimal design or optimal operation schedule wide range of stochastic systems. We also clarify the conditions for stochastic monotonicity of the Markov process. From the result it is shown that stachstic convexity can be used for the analysis of probabilitic models based on birth and death processes, which have very wide applications area. Finally we demonstrate the validity and usefulness of the theoretical results by developing efficient methods for the optimal replacement scheduling based on the stochastic convexity property.

• 한국정보통신학회논문지
• /
• 제8권6호
• /
• pp.1175-1181
• /
• 2004

본 논문에서는 비디오 검색을 위한 비디오 사진 분류 시스템을 제안하였다. 제안된 시스템은 3개의 모듈인 특징 추출, 은닉 마르코브 모델 생성, 그리고 비디오 사진 분류로 구성되어 있다. 같은 등급에 속한 비디오 화면들이 반드시 유사하지 않으므로 견실한 Hidden Markov Model을 구성하기 위해서 는 충분한 학습이 필요하였다. 제안된 시스템은 텔레비전 야구 중계 방송의 비디오 화면을 15가지 등급으로 분류하여 분석 및 하는 실험을 한 결과 평균 84.72%의 인식률을 얻을 수 있었다.

• International journal of advanced smart convergence
• /
• 제7권1호
• /
• pp.15-23
• /
• 2018

Spectrum allocation is a key operation in cognitive radio networks (CRNs), where secondary users (SUs) are usually selfish - to achieve itself utility maximization. In view of this context, much prior lit literature proposed spectrum allocation base on non-cooperative game models. However, the most of them proposed non-cooperative game models based on complete information of CRNs. In practical, primary users (PUs) in a dynamic wireless environment with noise uncertainty, shadowing, and fading is difficult to attain a complete information about them. In this paper, we propose a non-cooperative game joint hidden markov model scheme for spectrum allocation in CRNs. Firstly, we propose a new hidden markov model for SUs to predict the sensing results of competitors. Then, we introduce the proposed hidden markov model into the non-cooperative game. That is, it predicts the sensing results of competitors before the non-cooperative game. The simulation results show that the proposed scheme improves the energy efficiency of networks and utilization of SUs.

• Industrial Engineering and Management Systems
• /
• 제14권2호
• /
• pp.210-219
• /
• 2015

A classification task requires an exponentially growing amount of computation time and number of observations as the variable dimensionality increases. Thus, reducing the dimensionality of the data is essential when the number of observations is limited. Often, dimensionality reduction or feature selection leads to better classification performance than using the whole number of features. In this paper, we study the possibility of utilizing the Markov blanket discovery algorithm as a new feature selection method. The Markov blanket of a target variable is the minimal variable set for explaining the target variable on the basis of conditional independence of all the variables to be connected in a Bayesian network. We apply several Markov blanket discovery algorithms to some high-dimensional categorical and continuous data sets, and compare their classification performance with other feature selection methods using well-known classifiers.

• 국제학술발표논문집
• /
• The 9th International Conference on Construction Engineering and Project Management
• /
• pp.1043-1050
• /
• 2022

Markov chains and Monte Carlo Simulation were applied to account for the probabilistic nature of pavement deterioration over time using data collected in the field. The primary purpose of this study was to evaluate pavement network performance of Western Australia (WA) by applying the existing pavement management tools relevant to WA road construction networks. Two approaches were used to analyze the pavement networks: evaluating current pavement performance data to assess WA State Road networks and predicting the future states using past and current pavement data. The Markov chains process and Monte Carlo Simulation methods were used to predicting future conditions. The results indicated that Markov chains and Monte Carlo Simulation prediction models perform well compared to pavement performance data from the last four decades. The results also revealed the impact of design, traffic demand, and climate and construction standards on urban pavement performance. This study recommends an appropriate and effective pavement engineering management system for proper pavement design and analysis, preliminary planning, future pavement maintenance and rehabilitation, service life, and sustainable pavement construction functionality.

• 한국HCI학회:학술대회논문집
• /
• 한국HCI학회 2008년도 학술대회 1부
• /
• pp.744-748
• /
• 2008

본 논문에서는 주어진 예제 멜로디 데이터를 이용하여 효과적으로 새로운 곡을 작곡하는 시스템을 제안한다. 우리가 제안하는 기법은 k-차원 마르코프 체인을 이용하여 마디 단위의 음악 블록을 합성한다. 한마디 단위를 하나의 마르코프 체인의 상태로 취급할 경우 매우 많은 상태를 고려해야 하므로, 이를 계층적 클러스터링 기법을 통하여 학습이 용이한 정도로 상태를 줄인다. 예제 데이터의 각 음악 블록은 소속된 클러스터 번호의 시퀀스로 대체되어 학습 데이터로 사용된다. 학습된 마르코프 체인의 상태를 전이하면서 각 상태에 해당되는 클러스터의 음악 블록을 랜덤하게 선택하여 합성한다. 학습된 마르코프 체인은 효과적으로 예제 음악과 비슷하면서 새로운 곡을 생성할 수 있었다.