Browse > Article
http://dx.doi.org/10.5351/KJAS.2021.34.3.309

Doubly-robust Q-estimation in observational studies with high-dimensional covariates  

Lee, Hyobeen (Department of Statistics, Korea University)
Kim, Yeji (Department of Statistics, Korea University)
Cho, Hyungjun (Department of Statistics, Korea University)
Choi, Sangbum (Department of Statistics, Korea University)
Publication Information
The Korean Journal of Applied Statistics / v.34, no.3, 2021 , pp. 309-327 More about this Journal
Abstract
Dynamic treatment regimes (DTRs) are decision-making rules designed to provide personalized treatment to individuals in multi-stage randomized trials. Unlike classical methods, in which all individuals are prescribed the same type of treatment, DTRs prescribe patient-tailored treatments which take into account individual characteristics that may change over time. The Q-learning method, one of regression-based algorithms to figure out optimal treatment rules, becomes more popular as it can be easily implemented. However, the performance of the Q-learning algorithm heavily relies on the correct specification of the Q-function for response, especially in observational studies. In this article, we examine a number of double-robust weighted least-squares estimating methods for Q-learning in high-dimensional settings, where treatment models for propensity score and penalization for sparse estimation are also investigated. We further consider flexible ensemble machine learning methods for the treatment model to achieve double-robustness, so that optimal decision rule can be correctly estimated as long as at least one of the outcome model or treatment model is correct. Extensive simulation studies show that the proposed methods work well with practical sample sizes. The practical utility of the proposed methods is proven with real data example.
Keywords
double robustness; precision medicine; propensity score; Q-learning; super learner (SL); variable selection;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Hernan MA, Lanoy E, Costagliola D, and Robins JM (2006). Comparison of dynamic treatment regimes via inverse probability weighting, Basic & Clinical Pharmacology & Toxicology, 98, 237-242.   DOI
2 Bertsekas D and Tsitsiklis J (1996). Neuro-Dynamic Programming, Athena Scientific, Massachusetts.
3 Henderson R, Ansell P, and Alshibani D (2010). Regret-regression for optimal dynamic treatment regimes, Biometrics, 66, 1192-1201.   DOI
4 Kravitz RL, Duan N, and Braslow J (2004). Evidence-based medicine, heterogeneity of treatment effects and the trouble with averages, Milbank Quarterly, 82, 661-687.   DOI
5 Lee BK, Lessler J, and Stuart EA (2010). Improving propensity score weighting using machine learning, Statistics in Medicine, 29, 337-346.   DOI
6 Longford NT and Nelder JA (1999). Statistics versus statistical science in the regulatory process, Statistics in Medicine, 18, 2311-2320.   DOI
7 Lu W, Zhang HH, and Zeng D (2011). Variable selection for optimal treatment decision, Statistical Methods in Medical Research, 22, 493-504.   DOI
8 Murphy SA (2003). Optimal dynamic treatment regimes (with discussions), Journal of the Royal Statistical Society: Series B, 65, 331-366.   DOI
9 Murphy SA (2005). A generalization error for q-learning, Journal of Machine Learning Research, 6, 1073-1097.
10 Polley E, LeDell E, Kennedy C, Lendle S, and van der Laan M (2016). Package 'superlearner', Retrieved from: https://cran.r-project.org/web/packages/SuperLearner/SuperLearner.pdf
11 Ridgeway G, McCaffrey D, Morral A, Griffin BA, and Burgette L (2016). Twang: toolkit for weighting and analysis of nonequivalent groups.
12 Thall PF, Millikan RE, and Sung HG (2000). Evaluating multiple treatment courses in clinical trials, Statistics in Medicine, 19, 1011-1028.   DOI
13 Chakraborty B (2011). Dynamic treatment regimes for managing chronic health conditions: a statistical perspective, American Journal of Public Health, 101, 40-45.   DOI
14 Robins JM, Orellana L, and Rotnitzky A (2008). Estimation and extrapolation of optimal treatment and testing strategies, Statistics in Medicine, 27, 4678-4721.   DOI
15 Sorensen TI (1996). Which patients may be harmed by good treatments?, Lancet, 348, 351-352.   DOI
16 Tian L, Alizaden AA, Gentles AJ, and Tibshirani R (2014). A simple method for estimating interactions between a treatment and a large number of covariates, Journal of the American Statistical Association, 109, 1517-1532.   DOI
17 Tibshirani R (1996). Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society: Series B, 58, 267-288.
18 Van der Laan MJ, Polley EC, and Hubbard AE (2007). Super learner, Statistical Applications in Genetics and Molecular Biology, 6.
19 Wallace MP and Moodie EEM (2015). Doubly-robust dynamic treatment regimen estimation via weighted least squares, Biometrics, 71, 636-644.   DOI
20 Westreich D, Lessler J, and Funk MJ (2010). Propensity score estimation: neural networks, support vector machines, decision trees (CART) and meta-classifiers as alternatives to logistic regression, Journal of Clinical Epidemiology, 63, 826-833.   DOI
21 Chakraborty B, Murphy SA, and Strecher V (2010). Inference for non-regular parameters in optimal dynamic treatment regimes, Statistical Methods in Medical Research, 19, 317-343.   DOI
22 Zhao P and Yu B (2006). On model selection consistency of lasso, Journal of Machine Learning Research, 7, 2541-2563.
23 Pineau J, Bellemare MG, Rush AJ, Ghizaru A, and Murphy SA (2007). Constructing evidence-based treatment strategies using methods from computer science, Drug and Alcohol Dependence, 88, 52-60.
24 Arjas E and Saarela O (2010). Optimal dynamic regimes: presenting a case for predictive inference, The International Journal of Biostatistics, 6.
25 Bellman R (1957). Dynamic Programming, Princeton University Press, Princeton.
26 Breiman L, Friedman JH, Olshen RA, and Stone CJ (1984). Classification and Regression Trees, Wadsworth, California.
27 Gunter L, Zhu J, and Murphy SA (2011). Variable selection for qualitative interactions, Statistical Methodology, 8, 42-55.   DOI
28 Kereiakes DJ, Obenchain RL, Barber BL, et al. (2000). Abciximab provides cost-effective survival advantage in high-volume interventional practice, American Heart Journal, 140, 603-610.   DOI
29 Robins JM, Hernan MA, and Brumback B (2000). Marginal structural models and causal inference in epidemiology, Epidemiology, 11, 550-560.   DOI
30 Geng Y, Zhang HH, and Lu W (2015). On optimal treatment regimes selection for mean survival time, Statistics in Medicine, 34, 1169-1184.   DOI
31 Wallace MP, Moodie EEM, and Stephens DA (2017). Model validation and selection for personalized medicine using dynamic-weighted ordinary least squares, Statistical Methods in Medical Research, 26, 1641-1653.   DOI
32 Robins JM (2004). Optimal structural nested models for optimal sequential decisions. In Proceedings of the Second Seattle Symposium on Biostatistics, Springer, New York.
33 Therneau T, Atkinson B, and Ripley B (2014). Package 'Rpart', Retrieved from: http://cran.r-project.org/web/packages/rpart/rpart.pdf
34 Van der Laan MJ and Petersen ML (2007). Causal effect models for realistic individualized treatment and intention to treat rules, The International Journal of Biostatistics, 3.
35 Zou H (2006). The adaptive lasso and its oracle properties, Journal of the American Statistical Association, 101, 1418-1429.   DOI