• Title/Summary/Keyword: Markov decision-making

Search Result 31, Processing Time 0.025 seconds

Approximate Dynamic Programming Based Interceptor Fire Control and Effectiveness Analysis for M-To-M Engagement (근사적 동적계획을 활용한 요격통제 및 동시교전 효과분석)

  • Lee, Changseok;Kim, Ju-Hyun;Choi, Bong Wan;Kim, Kyeongtaek
    • Journal of the Korean Society for Aeronautical & Space Sciences
    • /
    • v.50 no.4
    • /
    • pp.287-295
    • /
    • 2022
  • As low altitude long-range artillery threat has been strengthened, the development of anti-artillery interception system to protect assets against its attacks will be kicked off. We view the defense of long-range artillery attacks as a typical dynamic weapon target assignment (DWTA) problem. DWTA is a sequential decision process in which decision making under future uncertain attacks affects the subsequent decision processes and its results. These are typical characteristics of Markov decision process (MDP) model. We formulate the problem as a MDP model to examine the assignment policy for the defender. The proximity of the capital of South Korea to North Korea border limits the computation time for its solution to a few second. Within the allowed time interval, it is impossible to compute the exact optimal solution. We apply approximate dynamic programming (ADP) approach to check if ADP approach solve the MDP model within processing time limit. We employ Shoot-Shoot-Look policy as a baseline strategy and compare it with ADP approach for three scenarios. Simulation results show that ADP approach provide better solution than the baseline strategy.

Performance Evaluations of Professional Baseball Players using DEA/OERA (DEA/OERA를 이용한 프로야구 선수들에 대한 성과 측정)

  • Lee, Deok-Joo;Yang, Won-Mo
    • IE interfaces
    • /
    • v.17 no.4
    • /
    • pp.440-449
    • /
    • 2004
  • The OERA(Offensive Earned-Run Average) is a methodology for the performance evaluation of baseball players, which is based on a well- known Markov chain model. The DEA(Data Envelopment Analysis) is an LP-based evaluation technique for performance analysis of DMUs (Decision Making Units), whose production activities are characterized by multiple inputs and outputs. In this paper, the performances of Korean professional baseball players are analytically evaluated using both OERA and DEA methods. We discuss methodological strengths and drawbacks of two kinds of baseball evaluation techniques, by comparing both results. Finally to overcome the shortcomings of both methods, we develop a new analytical approach for baseball evaluation by combining OERA with DEA.

Design and Elucidation of Integrated Forecasting Model for Information Factor Analysis (정보인자분석(情報因子分析)을 위한 통합예측(統合豫測)모델의 설계(設計) 및 해석(解析))

  • Kim, Hong-Jae;Lee, Tae-Hui
    • Journal of Korean Society for Quality Management
    • /
    • v.21 no.1
    • /
    • pp.181-189
    • /
    • 1993
  • Over the past two decades, forecasting has gained widespread acceptance as an integral part of business planning and decision making. Accurate forecasting is a prerequisite to successful planning. Accordingly, recent advances in forecasting techniques are of exceptional value to corporate planners. But most of forecasting mothods are reveal its limit and problem for precision and reliability duing to each relationship for raw data and possibility of explanation for each variable. Therefore, to construct the Integrated Forecasting Model(IFM) for Information Factor Analysis, it shoud be considered that whether law data has time lag and variables are explained. For this. following several method can be used : Least Square Method, Markov Process, Fibonacci series, Auto-Correlation, Cross-Correlation, Serial Correlation and Random Walk Theory. Thus, the unified property of these several functions scales the safety and growth of the system which may be varied time-to-time.

  • PDF

Measuring the Impact of Competition on Pricing Behaviors in a Two-Sided Market

  • Kim, Minkyung;Song, Inseong
    • Asia Marketing Journal
    • /
    • v.16 no.1
    • /
    • pp.35-69
    • /
    • 2014
  • The impact of competition on pricing has been studied in the context of counterfactual merger analyses where expected optimal prices in a hypothetical monopoly are compared with observed prices in an oligopolistic market. Such analyses would typically assume static decision making by consumers and firms and thus have been applied mostly to data obtained from consumer packed goods such as cereal and soft drinks. However such static modeling approach is not suitable when decision makers are forward looking. When it comes to the markets for durable products with indirect network effects, consumer purchase decisions and firm pricing decisions are inherently dynamic as they take into account future states when making purchase and pricing decisions. Researchers need to take into account the dynamic aspects of decision making both in the consumer side and in the supplier side for such markets. Firms in a two-sided market typically subsidize one side of the market to exploit the indirect network effect. Such pricing behaviors would be more prevalent in competitive markets where firms would try to win over the battle for standard. While such qualitative expectation on the relationship between pricing behaviors and competitive structures could be easily formed, little empirical studies have measured the extent to which the distinct pricing structure in two-sided markets depends on the competitive structure of the market. This paper develops an empirical model to measure the impact of competition on optimal pricing of durable products under indirect network effects. In order to measure the impact of exogenously determined competition among firms on pricing, we compare the equilibrium prices in the observed oligopoly market to those in a hypothetical monopoly market. In computing the equilibrium prices, we account for the forward looking behaviors of consumers and supplier. We first estimate a demand function that accounts for consumers' forward-looking behaviors and indirect network effects. And then, for the supply side, the pricing equation is obtained as an outcome of the Markov Perfect Nash Equilibrium in pricing. In doing so, we utilize numerical dynamic programming techniques. We apply our model to a data set obtained from the U.S. video game console market. The video game console market is considered a prototypical case of two-sided markets in which the platform typically subsidizes one side of market to expand the installed base anticipating larger revenues in the other side of market resulting from the expanded installed base. The data consist of monthly observations of price, hardware unit sales and the number of compatible software titles for Sony PlayStation and Nintendo 64 from September 1996 to August 2002. Sony PlayStation was released to the market a year before Nintendo 64 was launched. We compute the expected equilibrium price path for Nintendo 64 and Playstation for both oligopoly and for monopoly. Our analysis reveals that the price level differs significantly between two competition structures. The merged monopoly is expected to set prices higher by 14.8% for Sony PlayStation and 21.8% for Nintendo 64 on average than the independent firms in an oligopoly would do. And such removal of competition would result in a reduction in consumer value by 43.1%. Higher prices are expected for the hypothetical monopoly because the merged firm does not need to engage in the battle for industry standard. This result is attributed to the distinct property of a two-sided market that competing firms tend to set low prices particularly at the initial period to attract consumers at the introductory stage and to reinforce their own networks and eventually finally to dominate the market.

  • PDF

Exploring the Usage of the DEMATEL Method to Analyze the Causal Relations Between the Factors Facilitating Organizational Learning and Knowledge Creation in the Ministry of Education

  • Park, Sun Hyung;Kim, Il Soo;Lim, Seong Bum
    • International Journal of Contents
    • /
    • v.12 no.4
    • /
    • pp.31-44
    • /
    • 2016
  • Knowledge creation and management are regarded as critical success factors for an organization's survival in the knowledge era. As a process of knowledge acquisition and sharing, organizational learning mechanisms (OLMs) guide the learning function of organizations represented by its different learning activities. We examined a variety of learning processes that constitute OLMs. In this study, we aimed to capture the process and framework of OLMs and knowledge sharing and acquisition. Factors facilitating OLMs were investigated at three levels: individual, group, and organizational. The concept of an OLM has received some attention in the field of organizational learning, however, the relationship among the factors generating OLMs has not been empirically tested. As part of the ongoing discussion, we attempted a systemic approach for OLMs. OLMs can be represented by factors that are inherent to the organization's system; therefore, prior to empirically testing the OLM generating factor(s), evaluation of its organizational integration is required to determine effective treatment of each factor. Thus, we developed a framework to manage knowledge and proposed a method to numerically evaluate factors influencing the OLMs. Specifically, composite importance (CI) of the Decision-Making Trial and Evaluation Laboratory (DEMATEL) method was applied to explore the interaction effect of these factors based on systemic approach. The augmented matrix thus generated is expected to serve as a stochastic matrix of an absorbing Markov chain.

Phrase-based Topic and Sentiment Detection and Tracking Model using Incremental HDP

  • Chen, YongHeng;Lin, YaoJin;Zuo, WanLi
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.12
    • /
    • pp.5905-5926
    • /
    • 2017
  • Sentiments can profoundly affect individual behavior as well as decision-making. Confronted with the ever-increasing amount of review information available online, it is desirable to provide an effective sentiment model to both detect and organize the available information to improve understanding, and to present the information in a more constructive way for consumers. This study developed a unified phrase-based topic and sentiment detection model, combined with a tracking model using incremental hierarchical dirichlet allocation (PTSM_IHDP). This model was proposed to discover the evolutionary trend of topic-based sentiments from online reviews. PTSM_IHDP model firstly assumed that each review document has been composed by a series of independent phrases, which can be represented as both topic information and sentiment information. PTSM_IHDP model secondly depended on an improved time-dependency non-parametric Bayesian model, integrating incremental hierarchical dirichlet allocation, to estimate the optimal number of topics by incrementally building an up-to-date model. To evaluate the effectiveness of our model, we tested our model on a collected dataset, and compared the result with the predictions of traditional models. The results demonstrate the effectiveness and advantages of our model compared to several state-of-the-art methods.

Long-Term Arrival Time Estimation Model Based on Service Time (버스의 정차시간을 고려한 장기 도착시간 예측 모델)

  • Park, Chul Young;Kim, Hong Geun;Shin, Chang Sun;Cho, Yong Yun;Park, Jang Woo
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.6 no.7
    • /
    • pp.297-306
    • /
    • 2017
  • Citizens want more accurate forecast information using Bus Information System. However, most bus information systems that use an average based short-term prediction algorithm include many errors because they do not consider the effects of the traffic flow, signal period, and halting time. In this paper, we try to improve the precision of forecast information by analyzing the influencing factors of the error, thereby making the convenience of the citizens. We analyzed the influence factors of the error using BIS data. It is shown in the analyzed data that the effects of the time characteristics and geographical conditions are mixed, and that effects on halting time and passes speed is different. Therefore, the halt time is constructed using Generalized Additive Model with explanatory variable such as hour, GPS coordinate and number of routes, and we used Hidden Markov Model to construct a pattern considering the influence of traffic flow on the unit section. As a result of the pattern construction, accurate real-time forecasting and long-term prediction of route travel time were possible. Finally, it is shown that this model is suitable for travel time prediction through statistical test between observed data and predicted data. As a result of this paper, we can provide more precise forecast information to the citizens, and we think that long-term forecasting can play an important role in decision making such as route scheduling.

An Analysis on the Optimal Level of the Maintenance Float Using Absorbing Markov Chain (흡수 마코프 체인을 활용한 적정 M/F 재고 수준에 관한 연구)

  • Kim, Yong;Yoon, Bong-Kyoo
    • Journal of the military operations research society of Korea
    • /
    • v.34 no.2
    • /
    • pp.163-174
    • /
    • 2008
  • The military is an organization where reliability and availability take much more importance than in any other organization. And, in line with a recent trend of putting emphasis on 'system readiness', not only functions but also availability of a weapon system has become one of achievement targets. In this regard, the military keeps spares for important facility and equipment, which is called as Maintenance Float (M/F), in order to enhance reliability and availability in case of an unforeseen event. The military has calculated yearly M/F requirements based on the number of equipment and utilization rate. However, this method of calculation has failed to meet the intended targets of reliability and availability due to lack of consideration on the characteristics of equipment malfunctions and maintenance unit's capability. In this research, we present an analysis model that can be used to determine an optimal M/F inventory level based on queuing and absorbed Markov chain theories. And, we applied the new analysis model to come out with an optimal volume of K-1 tank M/F for the OO division, which serves as counterattack military unit. In our view, this research is valuable because, while using more tractable methodology compared to previous research, we present a new analysis model that can describe decision making process on M/F level more satisfactorily.

Recommendation System of University Major Subject based on Deep Reinforcement Learning (심층 강화학습 기반의 대학 전공과목 추천 시스템)

  • Ducsun Lim;Youn-A Min;Dongkyun Lim
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.23 no.4
    • /
    • pp.9-15
    • /
    • 2023
  • Existing simple statistics-based recommendation systems rely solely on students' course enrollment history data, making it difficult to identify classes that match students' preferences. To address this issue, this study proposes a personalized major subject recommendation system based on deep reinforcement learning (DRL). This system gauges the similarity between students based on structured data, such as the student's department, grade level, and course history. Based on this information, it recommends the most suitable major subjects by comprehensively considering information about each available major subject and evaluations of the student's courses. We confirmed that this DRL-based recommendation system provides useful insights for university students while selecting their major subjects, and our simulation results indicate that it outperforms conventional statistics-based recommendation systems by approximately 20%. In light of these results, we propose a new system that offers personalized subject recommendations by incorporating students' course evaluations. This system is expected to assist students significantly in finding major subjects that align with their preferences and academic goals.

An Empirical Study on Statistical Optimization Model for the Portfolio Construction of Sponsored Search Advertising(SSA) (키워드검색광고 포트폴리오 구성을 위한 통계적 최적화 모델에 대한 실증분석)

  • Yang, Hognkyu;Hong, Juneseok;Kim, Wooju
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.2
    • /
    • pp.167-194
    • /
    • 2019
  • This research starts from the four basic concepts of incentive incompatibility, limited information, myopia and decision variable which are confronted when making decisions in keyword bidding. In order to make these concept concrete, four framework approaches are designed as follows; Strategic approach for the incentive incompatibility, Statistical approach for the limited information, Alternative optimization for myopia, and New model approach for decision variable. The purpose of this research is to propose the statistical optimization model in constructing the portfolio of Sponsored Search Advertising (SSA) in the Sponsor's perspective through empirical tests which can be used in portfolio decision making. Previous research up to date formulates the CTR estimation model using CPC, Rank, Impression, CVR, etc., individually or collectively as the independent variables. However, many of the variables are not controllable in keyword bidding. Only CPC and Rank can be used as decision variables in the bidding system. Classical SSA model is designed on the basic assumption that the CPC is the decision variable and CTR is the response variable. However, this classical model has so many huddles in the estimation of CTR. The main problem is the uncertainty between CPC and Rank. In keyword bid, CPC is continuously fluctuating even at the same Rank. This uncertainty usually raises questions about the credibility of CTR, along with the practical management problems. Sponsors make decisions in keyword bids under the limited information, and the strategic portfolio approach based on statistical models is necessary. In order to solve the problem in Classical SSA model, the New SSA model frame is designed on the basic assumption that Rank is the decision variable. Rank is proposed as the best decision variable in predicting the CTR in many papers. Further, most of the search engine platforms provide the options and algorithms to make it possible to bid with Rank. Sponsors can participate in the keyword bidding with Rank. Therefore, this paper tries to test the validity of this new SSA model and the applicability to construct the optimal portfolio in keyword bidding. Research process is as follows; In order to perform the optimization analysis in constructing the keyword portfolio under the New SSA model, this study proposes the criteria for categorizing the keywords, selects the representing keywords for each category, shows the non-linearity relationship, screens the scenarios for CTR and CPC estimation, selects the best fit model through Goodness-of-Fit (GOF) test, formulates the optimization models, confirms the Spillover effects, and suggests the modified optimization model reflecting Spillover and some strategic recommendations. Tests of Optimization models using these CTR/CPC estimation models are empirically performed with the objective functions of (1) maximizing CTR (CTR optimization model) and of (2) maximizing expected profit reflecting CVR (namely, CVR optimization model). Both of the CTR and CVR optimization test result show that the suggested SSA model confirms the significant improvements and this model is valid in constructing the keyword portfolio using the CTR/CPC estimation models suggested in this study. However, one critical problem is found in the CVR optimization model. Important keywords are excluded from the keyword portfolio due to the myopia of the immediate low profit at present. In order to solve this problem, Markov Chain analysis is carried out and the concept of Core Transit Keyword (CTK) and Expected Opportunity Profit (EOP) are introduced. The Revised CVR Optimization model is proposed and is tested and shows validity in constructing the portfolio. Strategic guidelines and insights are as follows; Brand keywords are usually dominant in almost every aspects of CTR, CVR, the expected profit, etc. Now, it is found that the Generic keywords are the CTK and have the spillover potentials which might increase consumers awareness and lead them to Brand keyword. That's why the Generic keyword should be focused in the keyword bidding. The contribution of the thesis is to propose the novel SSA model based on Rank as decision variable, to propose to manage the keyword portfolio by categories according to the characteristics of keywords, to propose the statistical modelling and managing based on the Rank in constructing the keyword portfolio, and to perform empirical tests and propose a new strategic guidelines to focus on the CTK and to propose the modified CVR optimization objective function reflecting the spillover effect in stead of the previous expected profit models.