• Title/Summary/Keyword: markov model

Search Result 1,628, Processing Time 0.029 seconds

Reinforcement Learning for Minimizing Tardiness and Set-Up Change in Parallel Machine Scheduling Problems for Profile Shops in Shipyard (조선소 병렬 기계 공정에서의 납기 지연 및 셋업 변경 최소화를 위한 강화학습 기반의 생산라인 투입순서 결정)

  • So-Hyun Nam;Young-In Cho;Jong Hun Woo
    • Journal of the Society of Naval Architects of Korea
    • /
    • v.60 no.3
    • /
    • pp.202-211
    • /
    • 2023
  • The profile shops in shipyards produce section steels required for block production of ships. Due to the limitations of shipyard's production capacity, a considerable amount of work is already outsourced. In addition, the need to improve the productivity of the profile shops is growing because the production volume is expected to increase due to the recent boom in the shipbuilding industry. In this study, a scheduling optimization was conducted for a parallel welding line of the profile process, with the aim of minimizing tardiness and the number of set-up changes as objective functions to achieve productivity improvements. In particular, this study applied a dynamic scheduling method to determine the job sequence considering variability of processing time. A Markov decision process model was proposed for the job sequence problem, considering the trade-off relationship between two objective functions. Deep reinforcement learning was also used to learn the optimal scheduling policy. The developed algorithm was evaluated by comparing its performance with priority rules (SSPT, ATCS, MDD, COVERT rule) in test scenarios constructed by the sampling data. As a result, the proposed scheduling algorithms outperformed than the priority rules in terms of set-up ratio, tardiness, and makespan.

Optimal sensor placement for structural health monitoring based on deep reinforcement learning

  • Xianghao Meng;Haoyu Zhang;Kailiang Jia;Hui Li;Yong Huang
    • Smart Structures and Systems
    • /
    • v.31 no.3
    • /
    • pp.247-257
    • /
    • 2023
  • In structural health monitoring of large-scale structures, optimal sensor placement plays an important role because of the high cost of sensors and their supporting instruments, as well as the burden of data transmission and storage. In this study, a vibration sensor placement algorithm based on deep reinforcement learning (DRL) is proposed, which can effectively solve non-convex, high-dimensional, and discrete combinatorial sensor placement optimization problems. An objective function is constructed to estimate the quality of a specific vibration sensor placement scheme according to the modal assurance criterion (MAC). Using this objective function, a DRL-based algorithm is presented to determine the optimal vibration sensor placement scheme. Subsequently, we transform the sensor optimal placement process into a Markov decision process and employ a DRL-based optimization algorithm to maximize the objective function for optimal sensor placement. To illustrate the applicability of the proposed method, two examples are presented: a 10-story braced frame and a sea-crossing bridge model. A comparison study is also performed with a genetic algorithm and particle swarm algorithm. The proposed DRL-based algorithm can effectively solve the discrete combinatorial optimization problem for vibration sensor placements and can produce superior performance compared with the other two existing methods.

The Analysis of Successional Trends by Topographic Positions in the Natural Deciduous Forest of Mt. Chumbong (점봉산(點鳳産) 일대 천연활엽수림(天然闊葉樹林)의 지형적(地形的) 위치(位置)에 따른 천이(遷移) 경향(傾向) 분석(分析))

  • Lee, Won Sup;Kim, Ji Hong;Jin, Guang Ze
    • Journal of Korean Society of Forest Science
    • /
    • v.89 no.5
    • /
    • pp.655-665
    • /
    • 2000
  • Taking account of the structural variation on species composition by topography, the successional trends were comparatively analyzed for the three topographic positions (valley, mid-slope, and ridge) in the natural deciduous forest of Mt. Chumbong area. The analysis was based upon the subsequent process of generation replacement by understory saplings and seedlings over the overstory trees which will be eventually fallen down. This study adopted the plot sampling method, establishing twenty $20m{\times}20m$ quadrats and collecting vegetation and site data on each different topographic position. The transition matrix model, which was modified from the mathematical theory of Markov chain, was employed to analyze the successional trends and thereafter to predict the overstory species composition in the future for each different topographic position. In valley, the simulation indicated the remarkable decrease in the proportion of species composition of present dominants Quercus mongolica and Fraxinus mandshurica from current 23% and 21% to around 4% of each at the steady state, which is predicted to take less than 200 years. On the other hand, the proportion of such species as Abies holophylla, Acer mono, Tilia amurensis, and Ulmus laciniata will increase at the steady state. In mid-slope, the result showed the remarkable decrease in the proportion of Juglans mandshurica, Kalopanax pictus, and Tilia amurensis from current 15%, 8%, and 15% to 2%, 1%, and 5%, respectively, at steady state predicted to take more than 250 years. In ridge, the current dominant Quercus mongolica was predicted to be decreased dramatically from 58% to 8% at steady state which could be achieved about 200 years. On the contrary, the proportion of Acer mono and Tilia amurensis will be increased from current 4% and 3% to more than 20% and 40%, respectively, at the steady state. Overall results suggested that the study forest is more likely seral rather than climax community. Even though a lot of variation is inevitable due to various kinds of site and vegetation development, the study forest is considered to be more than 200 years away from the steady state or climax in terms of overstory species composition.

  • PDF

An Empirical Study on Statistical Optimization Model for the Portfolio Construction of Sponsored Search Advertising(SSA) (키워드검색광고 포트폴리오 구성을 위한 통계적 최적화 모델에 대한 실증분석)

  • Yang, Hognkyu;Hong, Juneseok;Kim, Wooju
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.2
    • /
    • pp.167-194
    • /
    • 2019
  • This research starts from the four basic concepts of incentive incompatibility, limited information, myopia and decision variable which are confronted when making decisions in keyword bidding. In order to make these concept concrete, four framework approaches are designed as follows; Strategic approach for the incentive incompatibility, Statistical approach for the limited information, Alternative optimization for myopia, and New model approach for decision variable. The purpose of this research is to propose the statistical optimization model in constructing the portfolio of Sponsored Search Advertising (SSA) in the Sponsor's perspective through empirical tests which can be used in portfolio decision making. Previous research up to date formulates the CTR estimation model using CPC, Rank, Impression, CVR, etc., individually or collectively as the independent variables. However, many of the variables are not controllable in keyword bidding. Only CPC and Rank can be used as decision variables in the bidding system. Classical SSA model is designed on the basic assumption that the CPC is the decision variable and CTR is the response variable. However, this classical model has so many huddles in the estimation of CTR. The main problem is the uncertainty between CPC and Rank. In keyword bid, CPC is continuously fluctuating even at the same Rank. This uncertainty usually raises questions about the credibility of CTR, along with the practical management problems. Sponsors make decisions in keyword bids under the limited information, and the strategic portfolio approach based on statistical models is necessary. In order to solve the problem in Classical SSA model, the New SSA model frame is designed on the basic assumption that Rank is the decision variable. Rank is proposed as the best decision variable in predicting the CTR in many papers. Further, most of the search engine platforms provide the options and algorithms to make it possible to bid with Rank. Sponsors can participate in the keyword bidding with Rank. Therefore, this paper tries to test the validity of this new SSA model and the applicability to construct the optimal portfolio in keyword bidding. Research process is as follows; In order to perform the optimization analysis in constructing the keyword portfolio under the New SSA model, this study proposes the criteria for categorizing the keywords, selects the representing keywords for each category, shows the non-linearity relationship, screens the scenarios for CTR and CPC estimation, selects the best fit model through Goodness-of-Fit (GOF) test, formulates the optimization models, confirms the Spillover effects, and suggests the modified optimization model reflecting Spillover and some strategic recommendations. Tests of Optimization models using these CTR/CPC estimation models are empirically performed with the objective functions of (1) maximizing CTR (CTR optimization model) and of (2) maximizing expected profit reflecting CVR (namely, CVR optimization model). Both of the CTR and CVR optimization test result show that the suggested SSA model confirms the significant improvements and this model is valid in constructing the keyword portfolio using the CTR/CPC estimation models suggested in this study. However, one critical problem is found in the CVR optimization model. Important keywords are excluded from the keyword portfolio due to the myopia of the immediate low profit at present. In order to solve this problem, Markov Chain analysis is carried out and the concept of Core Transit Keyword (CTK) and Expected Opportunity Profit (EOP) are introduced. The Revised CVR Optimization model is proposed and is tested and shows validity in constructing the portfolio. Strategic guidelines and insights are as follows; Brand keywords are usually dominant in almost every aspects of CTR, CVR, the expected profit, etc. Now, it is found that the Generic keywords are the CTK and have the spillover potentials which might increase consumers awareness and lead them to Brand keyword. That's why the Generic keyword should be focused in the keyword bidding. The contribution of the thesis is to propose the novel SSA model based on Rank as decision variable, to propose to manage the keyword portfolio by categories according to the characteristics of keywords, to propose the statistical modelling and managing based on the Rank in constructing the keyword portfolio, and to perform empirical tests and propose a new strategic guidelines to focus on the CTK and to propose the modified CVR optimization objective function reflecting the spillover effect in stead of the previous expected profit models.

Balanced DQDB Applying the System with Cyclic Service for a Fair MAC Procotol (공정한 MAC 프로토콜을 위해 순환서비스시스템을 적용한 평형 DQDB)

  • 류희삼;강준길
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.18 no.12
    • /
    • pp.1919-1927
    • /
    • 1993
  • A new MAC protocol has been proposed and analysed to relieve the unfairness problems exhibited by the basic version of the DQDB standard. DQDB MAC protocol has the unfairness problems in throughputs. message delay and so or. And when the slots are reused or the file transmissions takes long, the unfairness problems in the system become worse. The new access protocol proposed here, which of called the Balanced DQDB, guarantees a fair bandwidth distribution by using one bit of the dual bus network protocol and keeps up all characteristics of DQDB. the DQDB analysis model introduced by Wen Jing, et al, was considered to analyse a sequential balance distribution of solts. And the probabilities of the empty in operation mode were represented to determine the probabilities for busy bits to generate on each node of the bus using the Markov chain. Through the simulations. the performances of the proposed Balanced DQDB and that of the standard DQDB of the BWB mechanism were compared at the state that the values of the RQ or CD counter on each node varied dynamically. As the results, it is shown that the Balanced DQDB has the decrement of throughputs in upstream, but the numbers of the used empty slots at each node of the Balanced DQDB had more than that of the others because the Balanced DQDB has over 0.9 throughputs in the 70~80% nodes of total node and it has constant throughputs at each node. And there results were analogous to that of the analytical model.

  • PDF

Analysis of Signaling Load of Mobile IPv6 and Hierarchical Mobile IPv6 (Mobile IPv6와 Hierarchical Mobile IPv6의 시그널링 부하 분석)

  • Kong Ki-Sik;Song MoonBae;Hwang Chong-Sun
    • Journal of KIISE:Information Networking
    • /
    • v.32 no.4
    • /
    • pp.515-524
    • /
    • 2005
  • As the number of the mobile nodes (MNs) increases in the networks, the signaling traffic generated by mobility management for MNs will increase explosively, and such a phenomenon will probably affect overall network performance. In this paper, we propose a novel analytical approach using a continuous-time Markov chain model and hierarchical network model for the analysis on the signaling load of representative IPv6 mobility support Protocols such as Mobile IPv6 (MIPv6) and Hierarchical Mobile IPv6 (HMIPv6). According to these analytical modeling, this paper derives the various signaling costs, which are generated by an MN during its average domain residence time when MIPv6 and HMIPv6 are deployed under the same network architecture, respectively. In addition, based on these derived costs, we investigate the effects of various mobility/traffic-related parameters on the signaling costs generated by an MN under MIPv6 and HMIPv6. The analytical results show that as the average moving speed of an MN gets higher and the binding lifetime is set . to the larger value, and as its average packet arrival rate gets lower, the total signaling cost generated during its average domain residence time under HMIPv6 will get relatively lower than that under MIPv6, and that under the reverse conditions, the total signaling cost under MIPv6 will get relatively lower than that under HMIPv6.

English Phoneme Recognition using Segmental-Feature HMM (분절 특징 HMM을 이용한 영어 음소 인식)

  • Yun, Young-Sun
    • Journal of KIISE:Software and Applications
    • /
    • v.29 no.3
    • /
    • pp.167-179
    • /
    • 2002
  • In this paper, we propose a new acoustic model for characterizing segmental features and an algorithm based upon a general framework of hidden Markov models (HMMs) in order to compensate the weakness of HMM assumptions. The segmental features are represented as a trajectory of observed vector sequences by a polynomial regression function because the single frame feature cannot represent the temporal dynamics of speech signals effectively. To apply the segmental features to pattern classification, we adopted segmental HMM(SHMM) which is known as the effective method to represent the trend of speech signals. SHMM separates observation probability of the given state into extra- and intra-segmental variations that show the long-term and short-term variabilities, respectively. To consider the segmental characteristics in acoustic model, we present segmental-feature HMM(SFHMM) by modifying the SHMM. The SFHMM therefore represents the external- and internal-variation as the observation probability of the trajectory in a given state and trajectory estimation error for the given segment, respectively. We conducted several experiments on the TIMIT database to establish the effectiveness of the proposed method and the characteristics of the segmental features. From the experimental results, we conclude that the proposed method is valuable, if its number of parameters is greater than that of conventional HMM, in the flexible and informative feature representation and the performance improvement.

A study on the connected-digit recognition using MLP-VQ and Weighted DHMM (MLP-VQ와 가중 DHMM을 이용한 연결 숫자음 인식에 관한 연구)

  • Chung, Kwang-Woo;Hong, Kwang-Seok
    • Journal of the Korean Institute of Telematics and Electronics S
    • /
    • v.35S no.8
    • /
    • pp.96-105
    • /
    • 1998
  • The aim of this paper is to propose the method of WDHMM(Weighted DHMM), using the MLP-VQ for the improvement of speaker-independent connect-digit recognition system. MLP neural-network output distribution shows a probability distribution that presents the degree of similarity between each pattern by the non-linear mapping among the input patterns and learning patterns. MLP-VQ is proposed in this paper. It generates codewords by using the output node index which can reach the highest level within MLP neural-network output distribution. Different from the old VQ, the true characteristics of this new MLP-VQ lie in that the degree of similarity between present input patterns and each learned class pattern could be reflected for the recognition model. WDHMM is also proposed. It can use the MLP neural-network output distribution as the way of weighing the symbol generation probability of DHMMs. This newly-suggested method could shorten the time of HMM parameter estimation and recognition. The reason is that it is not necessary to regard symbol generation probability as multi-dimensional normal distribution, as opposed to the old SCHMM. This could also improve the recognition ability by 14.7% higher than DHMM, owing to the increase of small caculation amount. Because it can reflect phone class relations to the recognition model. The result of my research shows that speaker-independent connected-digit recognition, using MLP-VQ and WDHMM, is 84.22%.

  • PDF

Design and Implementation of a Real-Time Lipreading System Using PCA & HMM (PCA와 HMM을 이용한 실시간 립리딩 시스템의 설계 및 구현)

  • Lee chi-geun;Lee eun-suk;Jung sung-tae;Lee sang-seol
    • Journal of Korea Multimedia Society
    • /
    • v.7 no.11
    • /
    • pp.1597-1609
    • /
    • 2004
  • A lot of lipreading system has been proposed to compensate the rate of speech recognition dropped in a noisy environment. Previous lipreading systems work on some specific conditions such as artificial lighting and predefined background color. In this paper, we propose a real-time lipreading system which allows the motion of a speaker and relaxes the restriction on the condition for color and lighting. The proposed system extracts face and lip region from input video sequence captured with a common PC camera and essential visual information in real-time. It recognizes utterance words by using the visual information in real-time. It uses the hue histogram model to extract face and lip region. It uses mean shift algorithm to track the face of a moving speaker. It uses PCA(Principal Component Analysis) to extract the visual information for learning and testing. Also, it uses HMM(Hidden Markov Model) as a recognition algorithm. The experimental results show that our system could get the recognition rate of 90% in case of speaker dependent lipreading and increase the rate of speech recognition up to 40~85% according to the noise level when it is combined with audio speech recognition.

  • PDF

Automatic speech recognition using acoustic doppler signal (초음파 도플러를 이용한 음성 인식)

  • Lee, Ki-Seung
    • The Journal of the Acoustical Society of Korea
    • /
    • v.35 no.1
    • /
    • pp.74-82
    • /
    • 2016
  • In this paper, a new automatic speech recognition (ASR) was proposed where ultrasonic doppler signals were used, instead of conventional speech signals. The proposed method has the advantages over the conventional speech/non-speech-based ASR including robustness against acoustic noises and user comfortability associated with usage of the non-contact sensor. In the method proposed herein, 40 kHz ultrasonic signal was radiated toward to the mouth and the reflected ultrasonic signals were then received. Frequency shift caused by the doppler effects was used to implement ASR. The proposed method employed multi-channel ultrasonic signals acquired from the various locations, which is different from the previous method where single channel ultrasonic signal was employed. The PCA(Principal Component Analysis) coefficients were used as the features of ASR in which hidden markov model (HMM) with left-right model was adopted. To verify the feasibility of the proposed ASR, the speech recognition experiment was carried out the 60 Korean isolated words obtained from the six speakers. Moreover, the experiment results showed that the overall word recognition rates were comparable with the conventional speech-based ASR methods and the performance of the proposed method was superior to the conventional signal channel ASR method. Especially, the average recognition rate of 90 % was maintained under the noise environments.