Comparing the Performance of Three Severity Scoring Systems for ICU Patients: APACHE III, SAPS II, MPM II

중환자 중증도 평가도구의 타당도 평가 - APACHE III, SAPS II, MPM II

  • Kwon, Young-Dae (Department of Social and Preventive Medicine College of Medicine Sungkyunkwan University) ;
  • Hwang, Jeong-Hae (Research Department, Health Insurance Review Agency) ;
  • Kim, Eun-Kyung (School of Nursing, Eulji University)
  • 권영대 (성균관대학교 의과대학 사회의학교실) ;
  • 황정해 (건강보험심사평가원 조사연구실) ;
  • 김은경 (을지의과대학교 간호대학)
  • Published : 2005.09.01

Abstract

Objectives : To evaluate the predictive validity of three scoring systems; the acute physiology and chronic health evaluation(APACHE) III, simplified acute physiology score(SAPS) II, and mortality probability model(MPM) II systems in critically ill patients. Methods : A concurrent and retrospective study conducted by collecting data on consecutive patients admitted to the intensive care unit(ICU) including surgical, medical and coronary care unit between January 1, 2004, and March 31, 2004. Data were collected on 348 patients consecutively admitted to the ICU(aged 16 years or older, no transfer, ICU stay at least 8 hours). Three models were analyzed using logistic regression. Discrimination was assessed using receiver operating characteristic(ROC) curves, sensitivity, specificity, and correct classification rate. Calibration was assessed using the Lemeshow-Hosmer goodness of fit H-statistic. Results : For the APACHE III, SAPS II and MPM II systems, the area under the receiver operating characterist ic(ROC) curves were 0.981, 0.978, and 0.941 respectively. With a predicted risk of 0.5, the sensitivities for the APACHE III, SAPS II, and MPM II systems were 81.1, 79.2 and 71.7%, the specificities 98.3, 98.6, and 98.3%, and the correct classification rates 95.7, 95.7, and 94.3%, respectively. The SAPS II and APACHE III systems showed good calibrations(chi-squared H=2.5838 p=0.9577 for SAPS II, and chi-squared H=4.3761 p=0.8217 for APACHE III). Conclusions : The APACHE III and SAPS II systems have excellent powers of mortality prediction, and calibration, and can be useful tools for the quality assessment of intensive care units(ICUs).

Keywords

References

  1. Shim A. Clinical survey of patients of the Intensive care unit in Wonju Christian Hospital. Korean J Anesthesiol 1994; 1(27): 84-89 (Korean)
  2. Nunn JF, Milledge JS, Singaraya J. Survival of patients ventilated in an intensive therapy unit. BMJ 1979; 285: 1525
  3. Thibault GE, Mulley AG, Barnett GO. Medical intensive care: Indications, interventions and outcome. N Engl J Med 1980; 302: 938 https://doi.org/10.1056/NEJM198004243021703
  4. Cho KJ. The general analysis of the patients in intensive care unit by severity systems of an university hospital in Seoul [dissertation]. Koea: Seoul National University; 1991 (Korean)
  5. Knaus WA, Draper EA, Wagner DP, Zimmerm an JE. APACHE II: A severity of disease classification system. Crit Care Med 1985; 13(10): 818-829 https://doi.org/10.1097/00003246-198510000-00009
  6. Knaus WA, Wagner DP, Draper EA, Zimmerm an JE, Bergner M, Bastos PG. The APACHE III prognostic system: Risk prediction of hospital mortality for critically ill hospitalized adult. Chest 1991; 100(6): 1619-1636 https://doi.org/10.1378/chest.100.6.1619
  7. Rafkin HS, Hoyt JW. Objective data and quality assurance programs: current and future trends. Crit Care Clinics 1994; 10(1): 157
  8. Knaus WA, Zimmerman JE, Wagner DP. APACHE-acute physiology and chronic health evaluation : A physiologically based classificatio n system. Crit Care Med 1981; 9: 591-597 https://doi.org/10.1097/00003246-198108000-00008
  9. Shortell SM, Zimmerman JE, Rousseau DM. The performance of intensive care units: does good management make a difference? Med Care 1994; 32: 508-525 https://doi.org/10.1097/00005650-199405000-00009
  10. Hall JB, Schmid GA, Wood LD. Principle of critical care. New York: McGraw-Hill Inc. ; 1992. p. 551-557
  11. Cullen DJ, Chernow B. Predicting outcome in critically ill patients. Crit Care Med 1994; 22: 1345-1348 https://doi.org/10.1097/00003246-199409000-00001
  12. Lemeshow S, Teres D, Avrunin JS, Gage RW. Refining intensive care outcome prediction by using changing probabilities of mortality. Crit Care Med 1988; 16: 470-477 https://doi.org/10.1097/00003246-198805000-00002
  13. Le Gall JR, Loiral P, Alperovitch A, Glaser P, Granthil C, Mathieu D, Mercier P, Thomas R, Viller D. A simplified acute physiology score for ICU patients. Crit Care Med 1984 ; 12 : 975-977 https://doi.org/10.1097/00003246-198411000-00012
  14. Wong DT, Crofts SL, Gomez M. Evaluation of predictive ability of APACHE II system and hospital outcome in Canadian intensive care unit patients. Crit Care Med 1995; 23: 1177- 1183 https://doi.org/10.1097/00003246-199507000-00005
  15. Zimmerman JE. The APACHE III study design: Analytic plan for evaluation of severity and outcome. Crit Care Med 1989; 17: S169- 221 https://doi.org/10.1097/00003246-198912000-00001
  16. Sculier JP, Paesmans M, Markiewicz E, Berghmans T. Scoring systems in cancer patients admitted for an acute complication in a medical intensive care unit. Crit Care Med 2000; 28: 2786-2792 https://doi.org/10.1097/00003246-200008000-00018
  17. Lee DH, No MY, Kim BS. Interhospital comparison of outcome from intensive care unit with APACHE III scoring system. Korean J Prev Med 1994; 27(3): 437-445 (Korean)
  18. Lee KO, Shin HJ, Park HA, Jung HM, Lee MH, Choi EH, Lee JM. Kim UJ, Sim YK, Park GJ. Patient severity classification in a medical ICU using APACHE III and patient severity classification tool. J Korean Acad Nurs 2000; 30(5): 1243-1253 (Korean)
  19. Scneff M, Knaus WA. Predicting patient outcome from intensive care: A guide to APACHE, MPM, SAPS, PRISM and other prognostic scoring system. J Intensive Care Med 1990; 5(1): 33-52 https://doi.org/10.1177/088506669000500107
  20. Laurent GG, Turner MO, Andrew D. Rating the quality of intensive care units: Is it a function of the intensive care unit scoring system? Crit Care Med 2002; 30(9): 1976- 1982 https://doi.org/10.1097/00003246-200209000-00005
  21. Teres D, Lemeshow S, Aurunin JS. Validation of the mortality prediction model for ICU patients. Crit Care Med 1987; 15: 208-212 https://doi.org/10.1097/00003246-198703000-00005
  22. Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score(SAPS II) based on a European/North Americal multicenter study. JAMA 1993; 270: 2957- 2963 https://doi.org/10.1001/jama.270.24.2957
  23. Lemeshow S, Teres D, Klar J. Mortality probability models(MPM II) based on an international cohort of intensive care unit patients. JAMA 1993; 270: 2957-2963 https://doi.org/10.1001/jama.270.24.2957
  24. Sicignano A, Giudici D. Customization of SAPS II for the assessment of severity in Italian ICU patients. Minerva Anesthesiol 2000; 66: 139-145
  25. Rue M, Artigas A, Alvarez M. Performance of the mortality probability models(MPM II) in assessing severity of illness during the first week in the intensive care unit. Crit Care Med 2000; 28: 2819-2824 https://doi.org/10.1097/00003246-200008000-00023
  26. Beck DH, Taylor BL, Millar B. Prediction of outcome from intensive care: A prospective cohort study comparing APACHE II and III prognostic systems in a United Kingdom intensive care unit. Crit Care Med 1997; 25: 9- 15 https://doi.org/10.1097/00003246-199701000-00006
  27. Castella X, Artiga A, Bion J, Kari A. A comparison of severity of illness scoring systems for intensive care unit patients: results of multicenter, multinational study. Crit Care Med 1995; 23(8): 1327-1335 https://doi.org/10.1097/00003246-199508000-00005
  28. Castella K, Artigas A, Bion J, Kari A & The European/North American Severity Study Group. A comparison of severity of illness scoring systems for intensive care unit patients of multicenter, multinational study. Crit Car Med 1995; 23(8): 1327-1335 https://doi.org/10.1097/00003246-199508000-00005
  29. Patel PA, Grant BJB. Application of mortality prediction systems to individual intensive care units. Intensive Care Med 1999; 25: 977-982 https://doi.org/10.1007/s001340050992
  30. Koh BY. Prediction model for in-hospital mortality of patients visited emergency room with acute myocardial infarction [dissertation]. Korea: Chung-ang University; 2002 (Korean)
  31. Moreno R, Morians P. Outcome prediction in intensive care: Its of a prospective, multicenter, Portuguese study. Intensive Care Med 1997; 23: 177-186 https://doi.org/10.1007/s001340050313
  32. Reina A, Vazquez G, Aguayo E, Bravo I, Colmenero M, Bravo M. Mortality discriminat ion in acute myocardial infarction: comparison between APACHE III & SAPS II prognosis system. Intensive Care Med 1997; 23: 326-330 https://doi.org/10.1007/s001340050335
  33. Lee JH, Back KJ, Han SB, Ahn ST, Shin DW, Kim AJ, Kim JS. Mortality analysis of intensive care units patients using mortality probability models(MPM II). J Traumatol 2001; 14(2): 101-107 (Korean)
  34. Guiguet M, Blot F, escudier B, Antoun S, Leclercq B, Nitenber G. Severity illness scores for neutropenic patients in an intensive care unit: which is the best predictor? Do multiple assessment times improve the predictive value? Crit Care Med 1998; 26: 488-493 https://doi.org/10.1097/00003246-199803000-00020
  35. Schellongowski P, Benesch M, Lang T, Traunmuller F, Zauner C, Laczika K, Locker GJ, Frass M, Staudinge T. Comparison of three severity score for critically ill cancer patients. Intensive Care Med 2004; 30(3): 430-436 https://doi.org/10.1007/s00134-003-2043-1
  36. Iezzoni LI. The risk adjustment. JAMA 1997; 278(19): 1600-1607 https://doi.org/10.1001/jama.278.19.1600
  37. Park YS. A study on correlation of clinical process indicator and mortality after severity adjustment [dissertation]. Korea: Korea University; 1998 (Korean)
  38. Carson SS, Bach PB. Predicting mortality in patients suffering from prolonged critical illness: an assessment of four severity-of-illness measures. Chest 2001; 120: 928-933 https://doi.org/10.1378/chest.120.3.928
  39. Groeger JS, Lemeshow P, Nierman DM, White P Jr, Klar J, Granovsky S, Horak D, Kish SK. Multicenter outcome study of cancer patients admitted to the intensive care unit: a probability mortality model. J Clin Oncol 1998; 16: 761-770