Journal of Preventive Medicine and Public Health

The Korean Society for Preventive Medicine (대한예방의학회)
권호 표지
DOI QR코드

DOI QR Code

Improving the Performance of Risk-adjusted Mortality Modeling for Colorectal Cancer Surgery by Combining Claims Data and Clinical Data

  • Jang, Won Mo (People's Health Institution) ;
  • Park, Jae-Hyun (Department of Social and Preventive Medicine, Sungkyunkwan University) ;
  • Park, Jong-Hyock (Division of Cancer Policy and Management, National Cancer Control Research Institute, National Cancer Center) ;
  • Oh, Jae Hwan (Center for Colorectal Cancer, National Cancer Center) ;
  • Kim, Yoon (Institute of Health Policy and Management, Medical Research Center, Seoul National University)
  • Received : 2012.11.22
  • Accepted : 2013.02.19
  • Published : 2013.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

Objectives: The objective of this study was to evaluate the performance of risk-adjusted mortality models for colorectal cancer surgery. Methods: We investigated patients (n=652) who had undergone colorectal cancer surgery (colectomy, colectomy of the rectum and sigmoid colon, total colectomy, total proctectomy) at five teaching hospitals during 2008. Mortality was defined as 30-day or in-hospital surgical mortality. Risk-adjusted mortality models were constructed using claims data (basic model) with the addition of TNM staging (TNM model), physiological data (physiological model), surgical data (surgical model), or all clinical data (composite model). Multiple logistic regression analysis was performed to develop the risk-adjustment models. To compare the performance of the models, both c-statistics using Hanley-McNeil pair-wise testing and the ratio of the observed to the expected mortality within quartiles of mortality risk were evaluated to assess the abilities of discrimination and calibration. Results: The physiological model (c=0.92), surgical model (c=0.92), and composite model (c=0.93) displayed a similar improvement in discrimination, whereas the TNM model (c=0.87) displayed little improvement over the basic model (c=0.86). The discriminatory power of the models did not differ by the Hanley-McNeil test (p>0.05). Within each quartile of mortality, the composite and surgical models displayed an expected mortality ratio close to 1. Conclusions: The addition of clinical data to claims data efficiently enhances the performance of the risk-adjusted postoperative mortality models in colorectal cancer surgery. We recommended that the performance of models should be evaluated through both discrimination and calibration.

Keywords

References

  1. Steinberg SM, Popa MR, Michalek JA, Bethel MJ, Ellison EC. Comparison of risk adjustment methodologies in surgical quality improvement. Surgery 2008;144(4):662-667. https://doi.org/10.1016/j.surg.2008.06.010
  2. Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med 2007;356(5):486-496. https://doi.org/10.1056/NEJMsa064964
  3. Hall BL, Hirbe M, Waterman B, Boslaugh S, Dunagan WC. Comparison of mortality risk adjustment using a clinical data algorithm (American College of Surgeons National Surgical Quality Improvement Program) and an administrative data algorithm (Solucient) at the case level within a single institution. J Am Coll Surg 2007;205(6):767-777. https://doi.org/10.1016/j.jamcollsurg.2007.08.013
  4. Pine M, Jordan HS, Elixhauser A, Fry DE, Hoaglin DC, Jones B, et al. Enhancement of claims data to improve risk adjustment of hospital mortality. JAMA 2007;297(1):71-76. https://doi.org/10.1001/jama.297.1.71
  5. Shahian DM, Silverstein T, Lovett AF, Wolf RE, Normand SL. Comparison of clinical and administrative data sources for hospital coronary artery bypass graft surgery report cards. Circulation 2007;115(12):1518-1527. https://doi.org/10.1161/CIRCULATIONAHA.106.633008
  6. Gordon HS, Johnson ML, Wray NP, Petersen NJ, Henderson WG, Khuri SF, et al. Mortality after noncardiac surgery: prediction from administrative versus clinical data. Med Care 2005; 43(2):159-167. https://doi.org/10.1097/00005650-200502000-00009
  7. Park HK, Yoon SJ, Ahn HS, Ahn LS, Seo HJ, Lee SI, et al. Comparison of risk-adjustment models using administrative or clinical data for outcome prediction in patients after myocardial infarction or coronary bypass surgery in Korea. Int J Clin Pract 2007;61(7):1086-1090. https://doi.org/10.1111/j.1742-1241.2007.01345.x
  8. Scott IA, Thomson PL, Narasimhan S. Comparing risk-prediction methods using administrative or clinical data in assessing excess in-hospital mortality in patients with acute myocardial infarction. Med J Aust 2008;188(6):332-336.
  9. Fry DE, Pine M, Jordan HS, Elixhauser A, Hoaglin DC, Jones B, et al. Combining administrative and clinical data to stratify surgical risk. Ann Surg 2007;246(5):875-885. https://doi.org/10.1097/SLA.0b013e3180cc2e7a
  10. Pine M, Norusis M, Jones B, Rosenthal GE. Predictions of hospital mortality rates: a comparison of data sources. Ann Intern Med 1997;126(5):347-354. https://doi.org/10.7326/0003-4819-126-5-199703010-00002
  11. Geraci JM, Johnson ML, Gordon HS, Petersen NJ, Shroyer AL, Grover FL, et al. Mortality after cardiac bypass surgery: prediction from administrative versus clinical data. Med Care 2005; 43(2):149-158. https://doi.org/10.1097/00005650-200502000-00008
  12. Kang DB, Shin CY, Lee JK, Park WC. Multivariate analysis of the risk factors associated with complications and mortality after and emergency operation for obstructive, perforated colorectal cancer. J Korean Soc Coloproctol 2009;25(3):165-171 (Korean). https://doi.org/10.3393/jksc.2009.25.3.165
  13. Lee HJ, Oh JH, Lee JN, Baek JH, Chung M, Lee WK, et al. Prognostic factors associated with surgical mortality conferred by emergency operation in colorectal cancer. J Korean Soc Coloproctol 2006;22(5):301-307 (Korean).
  14. Park YJ, Park KJ, Park JG, Lee KU, Choe KJ, Kim JP. Prognostic factors in 2230 Korean colorectal cancer patients: analysis of consecutively operated cases. World J Surg 1999;23(7):721-726. https://doi.org/10.1007/PL00012376
  15. National Cancer Information Center. Incidence and mortality of cancer; 2012 [cited 2012 Dec 28]. Available from: http:// www.cancer.go.kr/ncic/cics_f/01/011/index.html (Korean).
  16. Ministry of Health and Welfare; National Cancer Center. Cancer facts and figures 2009. Seoul: Ministry of Health and Welfare; 2009, p. 2-28 (Korean).
  17. Obuchowski NA, McClish DK. Sample size determination for diagnostic accuracy studies involving binormal ROC curve indices. Stat Med 1997;16(13):1529-1542. https://doi.org/10.1002/(SICI)1097-0258(19970715)16:13<1529::AID-SIM565>3.0.CO;2-H
  18. Atherly A, Fink AS, Campbell DC, Mentzer RM Jr, Henderson W, Khuri S, et al. Evaluating alternative risk-adjustment strategies for surgery. Am J Surg 2004;188(5):566-570. https://doi.org/10.1016/j.amjsurg.2004.07.032
  19. Tourangeau AE, Tu JV. Developing risk-adjusted 30-day hospital mortality rates. Res Nurs Health 2003;26(6):483-496. https://doi.org/10.1002/nur.10110
  20. Tekkis PP, Poloniecki JD, Thompson MR, Stamatakis JD. Operative mortality in colorectal cancer: prospective national study. BMJ 2003;327(7425):1196-1201. https://doi.org/10.1136/bmj.327.7425.1196
  21. Aylin P, Bottle A, Majeed A. Use of administrative data or clinical databases as predictors of risk of death in hospital: comparison of models. BMJ 2007;334(7602):1044. https://doi.org/10.1136/bmj.39168.496366.55
  22. Mehrkhani F, Nasiri S, Donboli K, Meysamie A, Hedayat A. Prognostic factors in survival of colorectal cancer patients after surgery. Colorectal Dis 2009;11(2):157-161. https://doi.org/10.1111/j.1463-1318.2008.01556.x
  23. Liang H, Wang XN, Wang BG, Pan Y, Liu N, Wang DC, et al. Prognostic factors of young patients with colon cancer after surgery. World J Gastroenterol 2006;12(9):1458-1462. https://doi.org/10.3748/wjg.v12.i9.1458
  24. Tekkis PP, Prytherch DR, Kocher HM, Senapati A, Poloniecki JD, Stamatakis JD, et al. Development of a dedicated risk-adjustment scoring system for colorectal surgery (colorectal POSSUM). Br J Surg 2004;91(9):1174-1182. https://doi.org/10.1002/bjs.4430
  25. Rabeneck L, Souchek J, El-Serag HB. Survival of colorectal cancer patients hospitalized in the Veterans Affairs Health Care System. Am J Gastroenterol 2003;98(5):1186-1192. https://doi.org/10.1111/j.1572-0241.2003.07448.x
  26. Fazio VW, Tekkis PP, Remzi F, Lavery IC. Assessment of operative risk in colorectal cancer surgery: the Cleveland Clinic Foundation colorectal cancer model. Dis Colon Rectum 2004; 47(12):2015-2024. https://doi.org/10.1007/s10350-004-0704-y
  27. Nan KJ, Qin HX, Yang G. Prognostic factors in 165 elderly colorectal cancer patients. World J Gastroenterol 2003;9(10): 2207-2210. https://doi.org/10.3748/wjg.v9.i10.2207
  28. Longo WE, Virgo KS, Johnson FE, Oprian CA, Vernava AM, Wade TP, et al. Risk factors for morbidity and mortality after colectomy for colon cancer. Dis Colon Rectum 2000;43(1):83-91 https://doi.org/10.1007/BF02237249
  29. Compton CC, Fielding LP, Burgart LJ, Conley B, Cooper HS, Hamilton SR, et al. Prognostic factors in colorectal cancer. College of American Pathologists Consensus Statement 1999. Arch Pathol Lab Med 2000;124(7):979-994.
  30. Hadorn DC, Keeler EB, Rogers WH, Brook RH. Assessing the performance of mortality prediction models. Santa Monica: RAND; 1993, p. 1-32.
  31. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996; 15(4):361-387. https://doi.org/10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4
  32. Johnson RW. An introduction to the bootstrap. Teach Stat 2001;23(2):49-54. https://doi.org/10.1111/1467-9639.00050
  33. Iezzoni LI. Risk adjustment for measuring health care outcomes. 3rd ed. Chicago: Health Administration Press; 2003, p. 195-300.
  34. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148(3):839-843.
  35. Steyerberg EW, Eijkemans MJ, Harrell FE Jr, Habbema JD. Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. Stat Med 2000;19(8):1059-1079. https://doi.org/10.1002/(SICI)1097-0258(20000430)19:8<1059::AID-SIM412>3.0.CO;2-0
  36. Bilimoria KY, Bentrem DJ, Feinglass JM, Stewart AK, Winchester DP, Talamonti MS, et al. Directing surgical quality improvement initiatives: comparison of perioperative mortality and longterm survival for cancer surgery. J Clin Oncol 2008;26(28): 4626-4633. https://doi.org/10.1200/JCO.2007.15.6356
  37. Al-Homoud S, Purkayastha S, Aziz O, Smith JJ, Thompson MD, Darzi AW, et al. Evaluating operative risk in colorectal cancer surgery: ASA and POSSUM-based predictive models. Surg Oncol 2004;13(2-3):83-92. https://doi.org/10.1016/j.suronc.2004.08.006
  38. Tekkis PP, Kinsman R, Thompson MR, Stamatakis JD; Association of Coloproctology of Great Britain, Ireland. The Association of Coloproctology of Great Britain and Ireland study of large bowel obstruction caused by colorectal cancer. Ann Surg 2004;240(1):76-81. https://doi.org/10.1097/01.sla.0000130723.81866.75

Cited by

  1. Problems With Public Reporting of Cancer Quality Outcomes Data vol.10, pp.3, 2013, https://doi.org/10.1200/jop.2014.001405
  2. Model for risk adjustment of postoperative mortality in patients with colorectal cancer vol.102, pp.3, 2015, https://doi.org/10.1002/bjs.9696
  3. Variation between Hospitals with Regard to Diagnostic Practice, Coding Accuracy, and Case-Mix. A Retrospective Validation Study of Administrative Data versus Medical Records for Estimating 30-Day Mort vol.11, pp.5, 2013, https://doi.org/10.1371/journal.pone.0156075
  4. Modified Tumor Budding as a Better Predictor of Lymph Node Metastasis in Early Gastric Cancer: Possible Real-World Applications vol.13, pp.14, 2013, https://doi.org/10.3390/cancers13143405