• International Journal of Quality Innovation
• /
• 제2권2호
• /
• pp.136-144
• /
• 2001

The Markov Chain approach is used to develop an economic adjustment model of a process whose quality can be affected by a single special cause, resulting in changes of the process mean by incorrect adjustment of the process when it is operating according to its capability. The $\bar{X}$ control chart is thus used to signal the special cause. It is demonstrated that the expressions for the expected cycle time and the expected cycle cost are easier to obtain by the proposed approach than by adopting that in Collani, Saniga and Weigang (1994). Furthermore, this approach would be easily extended to derive the expected cycle cost and the expected cycle time for the case of multiple special causes or multiple control charts. A numerical example illustrates the proposed method and its application.

• 한국HCI학회:학술대회논문집
• /
• 한국HCI학회 2008년도 학술대회 1부
• /
• pp.739-743
• /
• 2008

본 연구에서는 사용자가 입력한 멜로디에 따른 반주 음악을 자동으로 생성하는 방법을 제시한다. 시작되는 코드는 사용자의 멜로디에 의해서 생성이 되며, 그 다음 코드들은 코드들간의 전이확률이 정의되어있는 마르코프 체인(markov chain)의 확률 테이블을 이용하여 연속적으로 생성된다. 확률 테이블은 기존 음악의 샘플 데이터를 강화학습(reinforcement learning)을 이용하여 학습된다. 또한 실시간으로 재생되는 반주 코드는 매 상태 마다 주어지는 보상 값을 통해 더 나은 행동을 취할 수 있도록 학습해 나간다. 멜로디와 각 코드들간의 유사성은 피치 클래스 히스토그램을 이용하여 계산된다. 본 기술을 사용하여 주어진 사용자 입력에 조화로운 반주 코드의 자동 생성이 가능하다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제10권8호
• /
• pp.3602-3620
• /
• 2016

Dynamic topology is one of the main influence factors on network performability. However, it was always ignored by the traditional network performability assessment methods when analyzing large-scale mobile ad hoc networks (MANETs) because of the state explosion problem. In this paper, we address this problem from the perspective of complex network. A two-layer hierarchical modeling approach is proposed for MANETs performability assessment, which can take both the dynamic topology and multi-state nodes into consideration. The lower level is described by Markov reward chains (MRC) to capture the multiple states of the nodes. The upper level is modeled as a small-world network to capture the characteristic path length based on different mobility and propagation models. The hierarchical model can promote the MRC of nodes into a state matrix of the whole network, which can avoid the state explosion in large-scale networks assessment from the perspective of complex network. Through the contrast experiments with OPNET simulation based on specific cases, the method proposed in this paper shows satisfactory performance on accuracy and efficiency.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제7권5호
• /
• pp.1036-1057
• /
• 2013

Wireless body area network (WBAN) is a promising candidate for future health monitoring system. Nevertheless, the path to mature solutions is still facing a lot of challenges that need to be overcome. Energy efficient scheduling is one of these challenges given the scarcity of available energy of biosensors and the lack of portability. Therefore, researchers from academia, industry and health sectors are working together to realize practical solutions for these challenges. The main difficulty in WBAN is the uncertainty in the state of the monitored system. Intelligent learning approaches such as a Markov Decision Process (MDP) were proposed to tackle this issue. A Markov Decision Process (MDP) is a form of Markov Chain in which the transition matrix depends on the action taken by the decision maker (agent) at each time step. The agent receives a reward, which depends on the action and the state. The goal is to find a function, called a policy, which specifies which action to take in each state, so as to maximize some utility functions (e.g., the mean or expected discounted sum) of the sequence of rewards. A partially Observable Markov Decision Processes (POMDP) is a generalization of Markov decision processes that allows for the incomplete information regarding the state of the system. In this case, the state is not visible to the agent. This has many applications in operations research and artificial intelligence. Due to incomplete knowledge of the system, this uncertainty makes formulating and solving POMDP models mathematically complex and computationally expensive. Limited progress has been made in terms of applying POMPD to real applications. In this paper, we surveyed the existing methods and algorithms for solving POMDP in the general domain and in particular in Wireless body area network (WBAN). In addition, the papers discussed recent real implementation of POMDP on practical problems of WBAN. We believe that this work will provide valuable insights for the newcomers who would like to pursue related research in the domain of WBAN.