• Title/Summary/Keyword: Markov process model

Search Result 368, Processing Time 0.022 seconds

Reinforcement Learning for Minimizing Tardiness and Set-Up Change in Parallel Machine Scheduling Problems for Profile Shops in Shipyard (조선소 병렬 기계 공정에서의 납기 지연 및 셋업 변경 최소화를 위한 강화학습 기반의 생산라인 투입순서 결정)

  • So-Hyun Nam;Young-In Cho;Jong Hun Woo
    • Journal of the Society of Naval Architects of Korea
    • /
    • v.60 no.3
    • /
    • pp.202-211
    • /
    • 2023
  • The profile shops in shipyards produce section steels required for block production of ships. Due to the limitations of shipyard's production capacity, a considerable amount of work is already outsourced. In addition, the need to improve the productivity of the profile shops is growing because the production volume is expected to increase due to the recent boom in the shipbuilding industry. In this study, a scheduling optimization was conducted for a parallel welding line of the profile process, with the aim of minimizing tardiness and the number of set-up changes as objective functions to achieve productivity improvements. In particular, this study applied a dynamic scheduling method to determine the job sequence considering variability of processing time. A Markov decision process model was proposed for the job sequence problem, considering the trade-off relationship between two objective functions. Deep reinforcement learning was also used to learn the optimal scheduling policy. The developed algorithm was evaluated by comparing its performance with priority rules (SSPT, ATCS, MDD, COVERT rule) in test scenarios constructed by the sampling data. As a result, the proposed scheduling algorithms outperformed than the priority rules in terms of set-up ratio, tardiness, and makespan.

Optimal sensor placement for structural health monitoring based on deep reinforcement learning

  • Xianghao Meng;Haoyu Zhang;Kailiang Jia;Hui Li;Yong Huang
    • Smart Structures and Systems
    • /
    • v.31 no.3
    • /
    • pp.247-257
    • /
    • 2023
  • In structural health monitoring of large-scale structures, optimal sensor placement plays an important role because of the high cost of sensors and their supporting instruments, as well as the burden of data transmission and storage. In this study, a vibration sensor placement algorithm based on deep reinforcement learning (DRL) is proposed, which can effectively solve non-convex, high-dimensional, and discrete combinatorial sensor placement optimization problems. An objective function is constructed to estimate the quality of a specific vibration sensor placement scheme according to the modal assurance criterion (MAC). Using this objective function, a DRL-based algorithm is presented to determine the optimal vibration sensor placement scheme. Subsequently, we transform the sensor optimal placement process into a Markov decision process and employ a DRL-based optimization algorithm to maximize the objective function for optimal sensor placement. To illustrate the applicability of the proposed method, two examples are presented: a 10-story braced frame and a sea-crossing bridge model. A comparison study is also performed with a genetic algorithm and particle swarm algorithm. The proposed DRL-based algorithm can effectively solve the discrete combinatorial optimization problem for vibration sensor placements and can produce superior performance compared with the other two existing methods.

Recognition for Noisy Speech by a Nonstationary AR HMM with Gain Adaptation Under Unknown Noise (잡음하에서 이득 적응을 가지는 비정상상태 자기회귀 은닉 마코프 모델에 의한 오염된 음성을 위한 인식)

  • 이기용;서창우;이주헌
    • The Journal of the Acoustical Society of Korea
    • /
    • v.21 no.1
    • /
    • pp.11-18
    • /
    • 2002
  • In this paper, a gain-adapted speech recognition method in noise is developed in the time domain. Noise is assumed to be colored. To cope with the notable nonstationary nature of speech signals such as fricative, glides, liquids, and transition region between phones, the nonstationary autoregressive (NAR) hidden Markov model (HMM) is used. The nonstationary AR process is represented by using polynomial functions with a linear combination of M known basis functions. When only noisy signals are available, the estimation problem of noise inevitably arises. By using multiple Kalman filters, the estimation of noise model and gain contour of speech is performed. Noise estimation of the proposed method can eliminate noise from noisy speech to get an enhanced speech signal. Compared to the conventional ARHMM with noise estimation, our proposed NAR-HMM with noise estimation improves the recognition performance about 2-3%.

Repeat Colonoscopy Every 10 Years or Single Colonoscopy for Colorectal Neoplasm Screening in Average-risk Chinese: A Cost-effectiveness Analysis

  • Wang, Zhen-Hua;Gao, Qin-Yan;Fang, Jing-Yuan
    • Asian Pacific Journal of Cancer Prevention
    • /
    • v.13 no.5
    • /
    • pp.1761-1766
    • /
    • 2012
  • Background: The appropriate interval between negative colonoscopy screenings is uncertain, but the numbers of advanced neoplasms 10 years after a negative result are generally low. We aimed to evaluate the cost-effectiveness of colorectal neoplasm screening and management based on repeat screening colonoscopy every 10 years or single colonoscopy, compared with no screening in the general population. Methods and materials: A state-transition Markov model simulated 100,000 individuals aged 50-80 years accepting repeat screening colonoscopy every 10 years or single colonoscopy, offered to every subject. Colorectal adenomas found during colonoscopy were removed by polypectomy, and the subjects were followed with surveillance every three years. For subjects with a normal result, colonoscopy was resumed within ten years in the repeat screening strategy. In single screening strategy, screening process was terminated. Direct costs such as screening tests, cancer treatment and costs of complications were included. Indirect costs were excluded from the model. The incremental cost-effectiveness ratio was used to evaluate the cost-effectiveness of the different screening strategies. Results: Assuming a first-time compliance rate of 90%, repeat screening colonoscopy and single colonoscopy can reduce the incidence of colorectal cancer by 65.8% and 67.2% respectively. The incremental cost-effectiveness ratio for single colonoscopy (49 Renminbi Yuan [RMB]) was much lower than that for repeat screening colonoscopy (474 RMB). Single colonoscopy was a more cost-effective strategy, which was not sensitive to the compliance rate of colonoscopy and the cost of advanced colorectal cancer. Conclusion: Single colonoscopy is suggested to be the more cost-effective strategy for screening and management of colorectal neoplasms and may be recommended in China clinical practice.

Variations of SST around Korea inferred from NOAA AVHRR data

  • Kang, Y. Q.;Hahn, S. D.;Suh, Y. S.;Park, S.J.
    • Proceedings of the KSRS Conference
    • /
    • 1998.09a
    • /
    • pp.236-241
    • /
    • 1998
  • The NOAA AVHRR remote sense SST data, collected by the National Fisheries Research and Development Institute (NFRDI), are analyzed in order to understand the spatial and temporal distributions of SST in the seas adjacent to Korea. Our study is based on 10-day SST images during last 7 years (1991-1997). For a time series analysis of multiple 557 images, all of images must be aligned exactly at the same position by adjusting the scales and positions of each SST image. We devised an algorithm which yields automatic detections of cloud pixels from multiple SST images. The cloud detection algorithm is based on a physical constraint that SST anomalies in the ocean do not exceed certain limits (we used $\pm$ 3$^{\circ}C$ as a criterion of SST anomalies). The remote sense SST data are tuned by comparing remote sense data with observed SST at coastal stations. Seasonal variations of SST are studied by harmonic fit of SST normals at each pixel. The SST anomalies are studied by statistical method. We found that the SST anomalies are rather persistent with time scales between 1 and 2 months. Utilizing the persistency of SST anomalies, we devised an algorithm for a prediction of future SST Model fit of SST anomalies to the Markov process model yields that autoregression coefficients of SST anomalies during a time elapse of 10 days are between 0.5 and 0.7. We plan to improve our algorithms of automatic cloud pixel detection and prediction of future SST. Our algorithm is expected to be incorporated to the operational real time service of SST around Korea.

  • PDF

A Comparison Study of Model Parameter Estimation Methods for Prognostics (건전성 예측을 위한 모델변수 추정방법의 비교)

  • An, Dawn;Kim, Nam Ho;Choi, Joo Ho
    • Journal of the Computational Structural Engineering Institute of Korea
    • /
    • v.25 no.4
    • /
    • pp.355-362
    • /
    • 2012
  • Remaining useful life(RUL) prediction of a system is important in the prognostics field since it is directly linked with safety and maintenance scheduling. In the physics-based prognostics, accurately estimated model parameters can predict the remaining useful life exactly. It, however, is not a simple task to estimate the model parameters because most real system have multivariate model parameters, also they are correlated each other. This paper presents representative methods to estimate model parameters in the physics-based prognostics and discusses the difference between three methods; the particle filter method(PF), the overall Bayesian method(OBM), and the sequential Bayesian method(SBM). The three methods are based on the same theoretical background, the Bayesian estimation technique, but the methods are distinguished from each other in the sampling methods or uncertainty analysis process. Therefore, a simple physical model as an easy task and the Paris model for crack growth problem are used to discuss the difference between the three methods, and the performance of each method evaluated by using established prognostics metrics is compared.

An Adaptive Load Control Scheme in Hierarchical Mobile IPv6 Networks (계층적 모바일 IP 망에서의 적응형 부하 제어 기법)

  • Pack Sang heon;Kwon Tae kyoung;Choi Yang hee
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.29 no.10A
    • /
    • pp.1131-1138
    • /
    • 2004
  • In Hierarchical Mobile Ipv6 (HMIPv6) networks, the mobility anchor point (MAP) handles binding update (BU) procedures locally to reduce signaling overhead for mobility. However, as the number of mobile nodes (MNs) handled by the MAP increases, the MAP suffers from the overhead not only to handle signaling traffic but also to Process data tunneling traffic. Therefore, it is important to control the number of MNs serviced by the MAP, in order to mitigate the burden of the MAP. We propose an adaptive load control scheme, which consists of two sub-algorithms: threshold-based admission control algorithm and session-to-mobility ratio (SMR) based replacement algorithm. When the number of MNs at a MAP reaches to the full capacity, the MAP replaces an existing MN at the MAP, whose SMR is high, with an MN that just requests binding update. The replaced MN is redirected to its home agent. We analyze the proposed load control scheme using the .Markov chain model in terms of the new MN and the ongoing MN blocking probabilities. Numerical results indicate that the above probabilities are lowered significantly compared to the threshold-based admission control alone.

Reverse link rate control for high-speed wireless systems based on traffic load prediction (고속 무선통신 시스템에서 트래픽 부하 예측에 의한 역방향 전송속도 제어)

  • Yeo, Woon-Young
    • Journal of the Institute of Electronics Engineers of Korea TC
    • /
    • v.45 no.11
    • /
    • pp.15-22
    • /
    • 2008
  • The cdma2000 1xEV-DO system controls the data rates of mobile terminals based on a binary overload indicator from the base station and a simple probabilistic model. However, this control scheme has difficulty in predicting the future behavior of mobile terminals due to a probabilistic uncertainty and has no reliable means of suppressing the traffic overload, which may result in performance degradation of CDMA systems that have interference-limited capacity. This Paper proposes a new traffic control scheme that controls the data rates of mobile terminals effectively by predicting the future traffic load and adjusting the forward-link control channel. The proposed scheme is analyzed by modeling it as a multi-dimensional Markov process and compared with conventional schemes. The numerical results show that the maximum cell throughput of the proposed scheme is much higher than those of the conventional schemes.

Design of the System and Algorithm for the Pattern Analysis of the Bio-Data (바이오 데이터 패턴 분석을 위한 시스템 및 알고리즘 설계)

  • Song, Young-Ohk;Kim, Sung-Young;Chang, Duk-Jin
    • The Journal of the Korea Contents Association
    • /
    • v.10 no.8
    • /
    • pp.104-110
    • /
    • 2010
  • In the field of biotechnology, computer can play varied roles such as the ordinal analysis, ordianl comparison, nutation tracing, analogy comparison for drug design, estimation of protein function, cell mechanism, and verifying the role of a gene for preventing diseases. Additionally, by constructing database, it can provide an application for the cloning process in other data researches, and be used as a basis for the comparative genetics. For the most of researcher about biotechnology, they need to use the tool that can do all of job above. This study is focused on looking into problems of existing systems to analysis bio data, and designing an improved analyzing system that can propose a solution. In additional, it has been considered to improve the performance of each constituent, and all the constituents, which have been separately processed, are combind in a single system to get over old problems of the existing system.

Traffic Flow Control of B-NT for Prevention of Congestion in B-ISDN UNI (B-ISDN UNI에서 폭주를 예방하기 위한 B-NT의 트래픽 흐름 제어)

  • 이숭희;최흥문
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.19 no.6
    • /
    • pp.1085-1094
    • /
    • 1994
  • We propose a traffic flow control scheme of B-NT with temporary cell buffering and selective cell discarding to prevent congestion state of the network nodes in B-ISDN systems to reduce or suppress output cell strams towards T interface. We define the states of the network nodes as normal, pre-congestion, and congestion. In a pre-congestion state, the loss-sensitive traffic is temporarily buffered to slow down the rate of the output traffic streams. In a congestion state, the delay-sensitive traffic is selectively discarded to suppress the output traffic streams as possible in addition to the cell buffering. We model the input cell streams and the states of the network nodes with Interrupted Bernoulli Process and 3-state Markov chain to analyze the performance of the proposed scheme in the B-NT system. The appropriate size of the cell buffer is explored by means of simulation and the influence on the performance of the proposed scheme by the network node state is discussed. As results, more than 2,00 cells of buffer size is needed for the control of medium of lower than the medium, degree of congestion occurrence in the network node while the control of high degree of congestion occurrence is nearly impossible.

  • PDF