DOI QR코드

DOI QR Code

How to Develop, Validate, and Compare Clinical Prediction Models Involving Radiological Parameters: Study Design and Statistical Methods

  • Han, Kyunghwa (Department of Radiology and Research Institute of Radiological Science, Severance Hospital, Yonsei University College of Medicine) ;
  • Song, Kijun (Department of Biostatistics and Medical Informatics, Yonsei University College of Medicine) ;
  • Choi, Byoung Wook (Department of Radiology and Research Institute of Radiological Science, Severance Hospital, Yonsei University College of Medicine)
  • 투고 : 2015.09.24
  • 심사 : 2016.01.14
  • 발행 : 2016.06.01

초록

Clinical prediction models are developed to calculate estimates of the probability of the presence/occurrence or future course of a particular prognostic or diagnostic outcome from multiple clinical or non-clinical parameters. Radiologic imaging techniques are being developed for accurate detection and early diagnosis of disease, which will eventually affect patient outcomes. Hence, results obtained by radiological means, especially diagnostic imaging, are frequently incorporated into a clinical prediction model as important predictive parameters, and the performance of the prediction model may improve in both diagnostic and prognostic settings. This article explains in a conceptual manner the overall process of developing and validating a clinical prediction model involving radiological parameters in relation to the study design and statistical methods. Collection of a raw dataset; selection of an appropriate statistical model; predictor selection; evaluation of model performance using a calibration plot, Hosmer-Lemeshow test and c-index; internal and external validation; comparison of different models using c-index, net reclassification improvement, and integrated discrimination improvement; and a method to create an easy-to-use prediction score system will be addressed. This article may serve as a practical methodological reference for clinical researchers.

키워드

참고문헌

  1. Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. New York: Springer, 2009
  2. D'Agostino RB Sr, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, et al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 2008;117:743-753 https://doi.org/10.1161/CIRCULATIONAHA.107.699579
  3. Yang HI, Yuen MF, Chan HL, Han KH, Chen PJ, Kim DY, et al. Risk estimation for hepatocellular carcinoma in chronic hepatitis B (REACH-B): development and validation of a predictive score. Lancet Oncol 2011;12:568-574 https://doi.org/10.1016/S1470-2045(11)70077-8
  4. Kwak JY, Jung I, Baek JH, Baek SM, Choi N, Choi YJ, et al. Image reporting and characterization system for ultrasound features of thyroid nodules: multicentric Korean retrospective study. Korean J Radiol 2013;14:110-117 https://doi.org/10.3348/kjr.2013.14.1.110
  5. Kim SY, Lee HJ, Kim YJ, Hur J, Hong YJ, Yoo KJ, et al. Coronary computed tomography angiography for selecting coronary artery bypass graft surgery candidates. Ann Thorac Surg 2013;95:1340-1346 https://doi.org/10.1016/j.athoracsur.2013.01.004
  6. Yoon YE, Lim TH. Current roles and future applications of cardiac CT: risk stratification of coronary artery disease. Korean J Radiol 2014;15:4-11 https://doi.org/10.3348/kjr.2014.15.1.4
  7. Shaw LJ, Giambrone AE, Blaha MJ, Knapper JT, Berman DS, Bellam N, et al. Long-term prognosis after coronary artery calcification testing in asymptomatic patients: a cohort study. Ann Intern Med 2015;163:14-21 https://doi.org/10.7326/M14-0612
  8. Lee K, Hur J, Hong SR, Suh YJ, Im DJ, Kim YJ, et al. Predictors of recurrent stroke in patients with ischemic stroke: comparison study between transesophageal echocardiography and cardiac CT. Radiology 2015;276:381-389 https://doi.org/10.1148/radiol.15142300
  9. Suh YJ, Hong YJ, Lee HJ, Hur J, Kim YJ, Lee HS, et al. Prognostic value of SYNTAX score based on coronary computed tomography angiography. Int J Cardiol 2015;199:460-466 https://doi.org/10.1016/j.ijcard.2015.07.067
  10. Sunshine JH, Applegate KE. Technology assessment for radiologists. Radiology 2004;230:309-314 https://doi.org/10.1148/radiol.2302031277
  11. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015;162:55-63 https://doi.org/10.7326/M14-0697
  12. Bossuyt PM, Leeflang MM. Chapter 6: Developing Criteria for Including Studies. In: Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 0.4 [updated September 2008]. Oxford: The Cochrane Collaboration, 2008
  13. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 1996;49:1373-1379 https://doi.org/10.1016/S0895-4356(96)00236-3
  14. Peduzzi P, Concato J, Feinstein AR, Holford TR. Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 1995;48:1503-1510 https://doi.org/10.1016/0895-4356(95)00048-8
  15. Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol 2007;165:710-718 https://doi.org/10.1093/aje/kwk052
  16. Steyerberg EW, Harrell FE Jr, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001;54:774-781 https://doi.org/10.1016/S0895-4356(01)00341-9
  17. Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med 2006;25:127-141 https://doi.org/10.1002/sim.2331
  18. Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ 1995;311:485 https://doi.org/10.1136/bmj.311.7003.485
  19. Austin PC, Tu JV. Automated variable selection methods for logistic regression produced unstable models for predicting acute myocardial infarction mortality. J Clin Epidemiol 2004;57:1138-1146 https://doi.org/10.1016/j.jclinepi.2004.04.003
  20. Austin PC, Tu JV. Bootstrap methods for developing predictive models. Am Stat 2004;58:131-137 https://doi.org/10.1198/0003130043277
  21. Austin PC. Bootstrap model selection had similar performance for selecting authentic and noise variables compared to backward variable elimination: a simulation study. J Clin Epidemiol 2008;61:1009-1017.e1 https://doi.org/10.1016/j.jclinepi.2007.11.014
  22. Little RJA, Rubin DB. Statistical analysis with missing data, 2nd ed. New York: John Wiley & Sons, 2014
  23. Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol 2004;159:882-890 https://doi.org/10.1093/aje/kwh101
  24. Nagelkerke NJ. A note on a general definition of the coefficient of determination. Biometrika 1991;78:691-692 https://doi.org/10.1093/biomet/78.3.691
  25. Tjur T. Coefficients of determination in logistic regression models-A new proposal: the coefficient of discrimination. Am Stat 2009;63;366-372 https://doi.org/10.1198/tast.2009.08210
  26. Rufibach K. Use of Brier score to assess binary predictions. J Clin Epidemiol 2010;63:938-939; author reply 939 https://doi.org/10.1016/j.jclinepi.2009.11.009
  27. Hosmer Jr DW, Lemeshow S. Applied logistic regression. New York: John Wiley & Sons, 2004
  28. D'Agostino R, Nam, BH. Evaluation of the performance of survival analysis models: discrimination and calibration measures. In: Balakrishnan N, Rao CO, eds. Handbook of statistics: advances in survival analysis. Vol 23. Amsterdam: Elsevier, 2004:1-25
  29. Hosmer DW, Hosmer T, Le Cessie S, Lemeshow S. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med 1997;16:965-980 https://doi.org/10.1002/(SICI)1097-0258(19970515)16:9<965::AID-SIM509>3.0.CO;2-O
  30. Park SH, Goo JM, Jo CH. Receiver operating characteristic (ROC) curve: practical review for radiologists. Korean J Radiol 2004;5:11-18 https://doi.org/10.3348/kjr.2004.5.1.11
  31. Harrell FE. Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis. New York: Springer, 2001
  32. Pencina MJ, D'Agostino RB Sr, Song L. Quantifying discrimination of Framingham risk functions with different survival C statistics. Stat Med 2012;31:1543-1553 https://doi.org/10.1002/sim.4508
  33. Van Calster B, Van Belle V, Vergouwe Y, Timmerman D, Van Huffel S, Steyerberg EW. Extending the c-statistic to nominal polytomous outcomes: the Polytomous Discrimination Index. Stat Med 2012;31:2610-2626 https://doi.org/10.1002/sim.5321
  34. Van Oirbeek R, Lesaffre E. An application of Harrell's C-index to PH frailty models. Stat Med 2010;29:3160-3171 https://doi.org/10.1002/sim.4058
  35. Wolbers M, Blanche P, Koller MT, Witteman JC, Gerds TA. Concordance for prognostic models with competing risks. Biostatistics 2014;15:526-539 https://doi.org/10.1093/biostatistics/kxt059
  36. Vergouwe Y, Steyerberg EW, Eijkemans MJ, Habbema JD. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol 2005;58:475-483 https://doi.org/10.1016/j.jclinepi.2004.06.017
  37. Collins GS, Ogundimu EO, Altman DG. Sample size considerations for the external validation of a multivariable prognostic model: a resampling study. Stat Med 2016;35:214-226 https://doi.org/10.1002/sim.6787
  38. Collins GS, de Groot JA, Dutton S, Omar O, Shanyinde M, Tajar A, et al. External validation of multivariable prediction models: a systematic review of methodological conduct and reporting. BMC Med Res Methodol 2014;14:40 https://doi.org/10.1186/1471-2288-14-40
  39. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837-845 https://doi.org/10.2307/2531595
  40. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007;115:928-935 https://doi.org/10.1161/CIRCULATIONAHA.106.672402
  41. Demler OV, Pencina MJ, D'Agostino RB Sr. Misuse of DeLong test to compare AUCs for nested models. Stat Med 2012;31:2577-2587 https://doi.org/10.1002/sim.5328
  42. Ware JH. The limitations of risk factors as prognostic tools. N Engl J Med 2006;355:2615-2617 https://doi.org/10.1056/NEJMp068249
  43. Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008;27:157-172; discussion 207-212 https://doi.org/10.1002/sim.2929
  44. Pepe MS. Problems with risk reclassification methods for evaluating prediction models. Am J Epidemiol 2011;173:1327-1335 https://doi.org/10.1093/aje/kwr013
  45. Leening MJ, Vedder MM, Witteman JC, Pencina MJ, Steyerberg EW. Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician's guide. Ann Intern Med 2014;160:122-131
  46. Pepe MS, Janes H. Commentary: reporting standards are needed for evaluations of risk reclassification. Int J Epidemiol 2011;40:1106-1108 https://doi.org/10.1093/ije/dyr083
  47. Widera C, Pencina MJ, Bobadilla M, Reimann I, Guba-Quint A, Marquardt I, et al. Incremental prognostic value of biomarkers beyond the GRACE (Global Registry of Acute Coronary Events) score and high-sensitivity cardiac troponin T in non-ST-elevation acute coronary syndrome. Clin Chem 2013;59:1497-1505 https://doi.org/10.1373/clinchem.2013.206185
  48. Pencina MJ, D'Agostino RB Sr, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med 2011;30:11-21 https://doi.org/10.1002/sim.4085
  49. Pepe MS, Kerr KF, Longton G, Wang Z. Testing for improvement in prediction model performance. Stat Med 2013;32:1467-1482 https://doi.org/10.1002/sim.5727
  50. Pepe MS, Janes H, Li CI. Net risk reclassification p values: valid or misleading? J Natl Cancer Inst 2014;106:dju041
  51. Kerr KF, McClelland RL, Brown ER, Lumley T. Evaluating the incremental value of new biomarkers with integrated discrimination improvement. Am J Epidemiol 2011;174:364-374 https://doi.org/10.1093/aje/kwr086
  52. Pencina MJ, D'Agostino RB, Pencina KM, Janssens AC, Greenland P. Interpreting incremental value of markers added to risk prediction models. Am J Epidemiol 2012;176:473-481 https://doi.org/10.1093/aje/kws207
  53. Sullivan LM, Massaro JM, D'Agostino RB Sr. Presentation of multivariate data for clinical use: The Framingham Study risk score functions. Stat Med 2004;23:1631-1660 https://doi.org/10.1002/sim.1742
  54. Imperiale TF, Monahan PO, Stump TE, Glowinski EA, Ransohoff DF. Derivation and Validation of a Scoring System to Stratify Risk for Advanced Colorectal Neoplasia in Asymptomatic Adults: A Cross-sectional Study. Ann Intern Med 2015;163:339-346 https://doi.org/10.7326/M14-1720
  55. Schnabel RB, Sullivan LM, Levy D, Pencina MJ, Massaro JM, D'Agostino RB Sr, et al. Development of a risk score for atrial fibrillation (Framingham Heart Study): a community-based cohort study. Lancet 2009;373:739-745 https://doi.org/10.1016/S0140-6736(09)60443-8
  56. Janssen KJ, Moons KG, Kalkman CJ, Grobbee DE, Vergouwe Y. Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol 2008;61:76-86 https://doi.org/10.1016/j.jclinepi.2007.04.018
  57. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies. Radiology 2015;277:826-832 https://doi.org/10.1148/radiol.2015151516
  58. Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015;162:W1-W73 https://doi.org/10.7326/M14-0698

피인용 문헌

  1. Statistical Assessment of Malaria Risk Factors Using Cox Proportional Hazard Approach vol.60, pp.2, 2016, https://doi.org/10.1080/09709274.2017.1387368
  2. Morphologic analysis with computed tomography may help differentiate fat-poor angiomyolipoma from renal cell carcinoma: a retrospective study with 602 patients vol.43, pp.3, 2016, https://doi.org/10.1007/s00261-017-1244-y
  3. Postnatal imaging of prenatally detected hydronephrosis-when is voiding cystourethrogram necessary? vol.33, pp.10, 2016, https://doi.org/10.1007/s00467-018-3938-y
  4. Methodologic Guide for Evaluating Clinical Performance and Effect of Artificial Intelligence Technology for Medical Diagnosis and Prediction vol.286, pp.3, 2018, https://doi.org/10.1148/radiol.2017171920
  5. CT Prediction Model for Major Arterial Injury after Blunt Pelvic Ring Disruption vol.287, pp.3, 2018, https://doi.org/10.1148/radiol.2018170997
  6. Health Outcomes and Lifestyle in a Sample of People With Multiple Sclerosis (HOLISM): Longitudinal and Validation Cohorts vol.9, pp.None, 2018, https://doi.org/10.3389/fneur.2018.01074
  7. Defining pneumonia after esophagectomy for cancer: validation of the Uniform Pneumonia Score in a high volume center in North America vol.31, pp.6, 2016, https://doi.org/10.1093/dote/doy002
  8. Web‐based thyroid imaging reporting and data system: Malignancy risk of atypia of undetermined significance or follicular lesion of undetermined significance thyroid nodules calculated by a comb vol.40, pp.9, 2016, https://doi.org/10.1002/hed.25173
  9. Correlates and aetiological factors associated with hedonic well-being among an ageing population of US men and women: secondary data analysis of a national survey vol.8, pp.11, 2018, https://doi.org/10.1136/bmjopen-2017-020962
  10. Predictors of quality of care and survival in a three-state cohort of locally advanced cervical cancer patients and development of a predictive model to identify women at risk of incomplete treatment vol.98, pp.33, 2016, https://doi.org/10.1097/md.0000000000016874
  11. Development and validation of an imaging and clinical scoring system to predict early mortality in spontaneous ruptured hepatocellular carcinoma treated with transarterial embolization vol.44, pp.3, 2016, https://doi.org/10.1007/s00261-019-01895-7
  12. Development and validation of a clinical model for preconception and early pregnancy risk prediction of gestational diabetes mellitus in nulliparous women vol.14, pp.4, 2016, https://doi.org/10.1371/journal.pone.0215173
  13. Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature vol.3, pp.None, 2016, https://doi.org/10.1186/s41512-019-0060-y
  14. A Scoring System for Prediction of Cervical Lymph Node Metastasis in Patients with Head and Neck Squamous Cell Carcinoma vol.40, pp.6, 2019, https://doi.org/10.3174/ajnr.a6066
  15. Risk scores identifying elderly inpatients at risk of 30-day unplanned readmission and accident and emergency department visit: a systematic review vol.9, pp.7, 2016, https://doi.org/10.1136/bmjopen-2018-028302
  16. Early Identification of Acute Traumatic Coagulopathy Using Clinical Prediction Tools: A Systematic Review vol.55, pp.10, 2016, https://doi.org/10.3390/medicina55100653
  17. Value of Computed Tomography Radiomic Features for Differentiation of Periprosthetic Mass in Patients With Suspected Prosthetic Valve Obstruction vol.12, pp.11, 2019, https://doi.org/10.1161/circimaging.119.009496
  18. Syndecan-1 Predicts Outcome in Patients with ST-Segment Elevation Infarction Independent from Infarct-related Myocardial Injury vol.9, pp.1, 2016, https://doi.org/10.1038/s41598-019-54937-x
  19. Development of a noninvasive tool to preoperatively evaluate the muscular invasiveness of bladder cancer using a radiomics approach vol.125, pp.24, 2016, https://doi.org/10.1002/cncr.32490
  20. Risk factors for aspiration pneumonia in patients with dysphagia undergoing videofluoroscopic swallowing studies : A retrospective cohort study vol.99, pp.46, 2016, https://doi.org/10.1097/md.0000000000023177
  21. Comment on “Prediction of the 1-Year Risk of Incident Lung Cancer: Prospective Study Using Electronic Health Records from the State of Maine” vol.22, pp.9, 2016, https://doi.org/10.2196/14944
  22. Radiomics and Deep Learning: Hepatic Applications vol.21, pp.4, 2016, https://doi.org/10.3348/kjr.2019.0752
  23. Automated contour detection in spine radiographs and computed tomography reconstructions for forensic comparative identification vol.16, pp.1, 2016, https://doi.org/10.1007/s12024-019-00189-0
  24. The multivariate predictive model to estimate ionized calcium concentration from serum biochemical results in dogs: External validation vol.49, pp.1, 2016, https://doi.org/10.1111/vcp.12835
  25. Development and Validation of a Risk Prediction Tool to Identify People with HIV Infection Likely Not to Achieve Viral Suppression vol.34, pp.4, 2020, https://doi.org/10.1089/apc.2019.0224
  26. Prediction models for prostate cancer to be used in the primary care setting: a systematic review vol.10, pp.7, 2016, https://doi.org/10.1136/bmjopen-2019-034661
  27. Optimizing the measurement of comorbidity for a South Australian colorectal cancer population using administrative data vol.26, pp.4, 2020, https://doi.org/10.1111/jep.13305
  28. Radiomics prognostication model in glioblastoma using diffusion- and perfusion-weighted MRI vol.10, pp.None, 2020, https://doi.org/10.1038/s41598-020-61178-w
  29. Clinical predictive score for detecting nonalcoholic fatty liver disease with significant fibrosis in patients with metabolic syndrome vol.100, pp.44, 2021, https://doi.org/10.1097/md.0000000000027640
  30. Machine Learning Applications in Mental Health and Substance Use Research Among the LGBTQ2S+ Population: Scoping Review vol.9, pp.11, 2021, https://doi.org/10.2196/28962
  31. A Single-Center Retrospective Analysis of Periprocedural Variables Affecting Local Tumor Progression after Radiofrequency Ablation of Colorectal Cancer Liver Metastases vol.298, pp.1, 2021, https://doi.org/10.1148/radiol.2020200109
  32. Periprocedural risk factors for incomplete radiofrequency ablation of liver metastases from colorectal cancer: a single-center retrospective analysis vol.38, pp.1, 2021, https://doi.org/10.1080/02656736.2021.1942564
  33. Predictive Value of Urinalysis and Recent Antibiotic Exposure to Distinguish Between Bacteriuria, Candiduria, and No-Growth Urine vol.14, pp.None, 2016, https://doi.org/10.2147/idr.s343021
  34. A Predictive Model for the Risk of Cognitive Impairment in Patients with Gallstones vol.2021, pp.None, 2016, https://doi.org/10.1155/2021/3792407
  35. A Diagnostic Nomogram Based on 18F-FDG PET/CT for Bone Metastasis of Gastric Cancer vol.9, pp.None, 2016, https://doi.org/10.3389/fcell.2021.783466
  36. Vessel Type Determined by Vessel Architectural Imaging Improves Differentiation between Early Tumor Progression and Pseudoprogression in Glioblastoma vol.42, pp.4, 2016, https://doi.org/10.3174/ajnr.a6984
  37. Outcomes prediction in longitudinal data: Study designs evaluation, use case in ICU acquired sepsis vol.117, pp.None, 2016, https://doi.org/10.1016/j.jbi.2021.103734
  38. Preoperative MRI-based estimation of risk for positive resection margin after radical prostatectomy in patients with prostate cancer: development and validation of a simple scoring system vol.31, pp.7, 2016, https://doi.org/10.1007/s00330-020-07569-z
  39. Development and Validation of a Nomogram for Predicting the Risk of Pregnancy-Induced Hypertension: A Retrospective Cohort Study vol.30, pp.8, 2021, https://doi.org/10.1089/jwh.2020.8575
  40. Development of a risk score to predict abnormal glycemic status among Thai dental patients vol.35, pp.5, 2021, https://doi.org/10.1108/jhr-09-2019-0213
  41. Development of a risk prediction nomogram for disposition of acute toxic exposure patients to intensive care unit vol.129, pp.3, 2021, https://doi.org/10.1111/bcpt.13619
  42. Fluorine-18 fluorodeoxyglucose positron emission tomography-based textural features for prediction of event prone carotid atherosclerotic plaques vol.28, pp.5, 2016, https://doi.org/10.1007/s12350-019-01943-1
  43. A risk-prediction model using parameters of maternal body composition to identify gestational diabetes mellitus in early pregnancy vol.45, pp.None, 2021, https://doi.org/10.1016/j.clnesp.2021.08.002
  44. Why did European Radiology reject my radiomic biomarker paper? How to correctly evaluate imaging biomarkers in a clinical setting vol.31, pp.12, 2016, https://doi.org/10.1007/s00330-021-07971-1
  45. Development of a novel nomogram-based online tool to predict axillary status after neoadjuvant chemotherapy in cN+ breast cancer: A multicentre study on 1,950 patients vol.60, pp.None, 2016, https://doi.org/10.1016/j.breast.2021.09.013
  46. Nomogram to Assess the Risk of Central Cervical Lymph Node Metastasis in Patients With Clinical N0 Papillary Thyroid Carcinoma vol.27, pp.12, 2021, https://doi.org/10.1016/j.eprac.2021.06.010
  47. Extended application of a CT-based artificial intelligence prognostication model in patients with primary lung cancer undergoing stereotactic ablative radiotherapy vol.165, pp.None, 2016, https://doi.org/10.1016/j.radonc.2021.10.022
  48. Prediction of ipsilateral lateral cervical lymph node metastasis in papillary thyroid carcinoma: a combined dual-energy CT and thyroid function indicators study vol.21, pp.1, 2016, https://doi.org/10.1186/s12885-021-07951-0