Journal of Preventive Medicine and Public Health

The Korean Society for Preventive Medicine (대한예방의학회)
권호 표지
DOI QR코드

DOI QR Code

Reliability and Validity of the Appropriateness Evaluation Protocol for Public Hospitals in Korea

  • Lee, Clara (Department of Thoracic and Cardiovascular Surgery, Asan Medical Center, University of Ulsan College of Medicine) ;
  • Kim, Stella Jung-Hyun (Department of Preventive Medicine, College of Medicine, The Catholic University of Korea) ;
  • Lee, Changwoo (School of Public Health, The Catholic University of Korea) ;
  • Shin, Euichul (Department of Preventive Medicine, College of Medicine, The Catholic University of Korea)
  • Received : 2019.05.27
  • Accepted : 2019.09.04
  • Published : 2019.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Objectives: This study was conducted to assess the applicability of the Appropriateness Evaluation Protocol (AEP) for public hospitals in Korea. Methods: In May 2016, 1500 admission claims were collected from Korean public district hospitals using stratified random sampling. Of these claims, 560 admissions to 37 hospitals were retrieved for analysis. Medical records administrators determined the appropriateness of admission using the criteria detailed in the AEP, and a physician separately assessed the appropriateness of admission based on her clinical judgment. To examine the applicability of the AEP, the concordance of the decisions made between a pair of AEP reviewers and between an AEP reviewer and a physician reviewer was compared. Results: The results showed an almost perfect inter-rater agreement between the AEP reviewers and a moderate agreement between the AEP reviewers and the physician. The sensitivity and specificity of the AEP were calculated as 0.86 and 0.56, respectively. Conclusions: Our findings suggest that the AEP could potentially be applied to Korean public hospitals as a reliable and valid instrument for assessing the appropriateness of admissions.

Keywords

References

  1. Donabedian A. The quality of care. How can it be assessed? JAMA 1988;260(12):1743-1748. https://doi.org/10.1001/jama.1988.03410120089033
  2. Lohr KN, Schroeder SA. A strategy for quality assurance in Medicare. N Engl J Med 1990;322(10):707-712. https://doi.org/10.1056/NEJM199003083221031
  3. Change Healthcare. $InterQual^{(R)}$ level of care criteria [cited 2019 Mar 5]. Available from: https://www.changehealthcare. com/solutions/interqual/level-of-care-criteria.
  4. Gertman PM, Restuccia JD. The appropriateness evaluation protocol:a technique for assessing unnecessary days of hospital care. Med Care 1981;19(8):855-871. https://doi.org/10.1097/00005650-198108000-00005
  5. Oak Group. Criteria [cited 2019 Mar 7]. Available from: https://oakgroup.com/mcap-clinical-utilisation-review/clinical-utilisation-review-criteria/.
  6. Esmaili A, Ravaghi H, Seyedin H, Delgoshaei B, Salehi M. Developing of the appropriateness evaluation protocol for public hospitals in Iran. Iran Red Crescent Med J 2015;17(3):e19030.
  7. Liu W, Yuan S, Wei F, Yang J, Zhang Z, Zhu C, et al. Reliability and validity of the Chinese version appropriateness evaluation protocol. PLoS One 2015;10(8):e0136498. https://doi.org/10.1371/journal.pone.0136498
  8. Lorenzo S, Lang T, Pastor R, Tampieri A, Santos-Eggimann B, Smith H, et al. Reliability study of the European appropriateness evaluation protocol. Int J Qual Health Care 1999;11(5):419-424. https://doi.org/10.1093/intqhc/11.5.419
  9. Robain M, Lang T, Fontaine A, Logerot H, Monnet E, Six P, et al. Reliability and validity of the French version of the first part of the appropriateness evaluation protocol (AEPf): pertinent criteria of hospitalization stay. Rev Epidemiol Sante Publique 1999;47(2):139-149.
  10. Apolone G, Fellin G, Tampieri A, Bonanoni E, Crosti PF, Lanzi E, et al. Appropriateness of hospital use: report from an Italian study. Eur J Public Health 1997;7(1):34-39. https://doi.org/10.1093/eurpub/7.1.34
  11. Kaya S, Vural G, Eroglu K, Sain G, Mersin H, Karabeyoglu M, et al. Liability and validity of the appropriateness evaluation protocol in Turkey. Int J Qual Health Care 2000;12(4):325-329. https://doi.org/10.1093/intqhc/12.4.325
  12. Lang T, Liberati A, Tampieri A, Fellin G, Gonsalves Mda L, Lorenzo S, et al. A European version of the appropriateness evaluation protocol: goals and presentation. Int J Technol Assess Health Care 1999;15(1):185-197. https://doi.org/10.1017/S0266462399152784
  13. Jepsen HK, Hendriksen C, Nielsen H, Nybo B, Perrild H. Every seventh acute medical admission is preventable. Dan Med J 2013;60(3):A4595.
  14. Panis LJ, Verheggen FW, Pop P. To stay or not to stay. The assessment of appropriate hospital stay: a Dutch report. Int J Qual Health Care 2002;14(1):55-67. https://doi.org/10.1093/intqhc/14.1.55
  15. Sangha O, Schneeweiss S, Wildner M, Cook EF, Brennan TA, Witte J, et al. Metric properties of the appropriateness evaluation protocol and predictors of inappropriate hospital use in Germany: an approach using longitudinal patient data. Int J Qual Health Care 2002;14(6):483-492. https://doi.org/10.1093/intqhc/14.6.483
  16. Santos-Eggimann B, Paccaud F, Blanc T. Medical appropriateness of hospital utilization: an overview of the Swiss experience. Int J Qual Health Care 1995;7(3):227-232. https://doi.org/10.1016/1353-4505(95)00017-P
  17. Sanchez-Garcia S, Juarez-Cedillo T, Mould-Quevedo JF, Garcia-Gonzalez JJ, Contreras-Hernandez I, Espinel-Bermudez MC, et al. The hospital appropriateness evaluation protocol in elderly patients: a technique to evaluate admission and hospital stay. Scand J Caring Sci 2008;22(2):306-313. https://doi.org/10.1111/j.1471-6712.2007.00528.x
  18. Esmail A. Development of the paediatric appropriateness evaluation protocol for use in the United Kingdom. J Public Health Med 2000;22(2):224-230. https://doi.org/10.1093/pubmed/22.2.224
  19. Chan SC. Appropriateness of medical admissions from a Malaysian public primary care clinic. Med J Malaysia 1999;54(3):329-337.
  20. Leung LP, Fan KL. Who should be admitted to hospital? Evaluation of a screening tool. Hong Kong Med J 2008;14(4):273-277.
  21. Hwang JI. Characteristics of patient and healthcare service utilization associated with inappropriate hospitalization days. J Adv Nurs 2007;60(6):654-662. https://doi.org/10.1111/j.1365-2648.2007.04452.x
  22. Yoem HY, Kim SL. Appropriateness evaluation of hospitalization for the cerebral ischemia patients. J Korean Acad Community Health Nurs 1999;10(1):80-92 (Korean).
  23. Oh J. Modernization of public district general hospitals: 2011 modularization of Korea's development experience. Seoul: Ministry of Strategy and Finance; 2012, p. 19-21.
  24. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159-174. https://doi.org/10.2307/2529310
  25. Tavakoli N, Hosseini Kasnavieh SM, Yasinzadeh M, Amini M, Mahmoudi Nejad M. Evaluation of appropriate and inappropriate admission and hospitalization days according to appropriateness evaluation protocol (AEP). Arch Iran Med 2015;18(7):430-434.
  26. Gamper G, Wiedermann W, Barisonzo R, Stockner I, Wiedermann CJ. Inappropriate hospital admission: interaction between patient age and co-morbidity. Intern Emerg Med 2011;6(4):361-367. https://doi.org/10.1007/s11739-011-0629-0
  27. Menand E, Lenain E, Lazarovici C, Chatellier G, Saint-Jean O, Somme D, et al. French multicenter evaluation of the appropriateness of admission to the emergency department of the over-80s. J Nutr Health Aging 2015;19(6):681-687. https://doi.org/10.1007/s12603-015-0489-9
  28. Andrew MK, Powell C. An approach to 'the social admission'. Can J Gen Intern Med 2015;10(4):20-22. https://doi.org/10.19144/1911-1606.10.1.2
  29. Mould-Quevedo JF, Garcia-Pena C, Contreras-Hernandez I, Juarez-Cedillo T, Espinel-Bermudez C, Morales-Cisneros G, et al. Direct costs associated with the appropriateness of hospital stay in elderly population. BMC Health Serv Res 2009;9:151. https://doi.org/10.1186/1472-6963-9-151
  30. Rojas-Garcia A, Turner S, Pizzo E, Hudson E, Thomas J, Raine R. Impact and experiences of delayed discharge: a mixed-studies systematic review. Health Expect 2018;21(1):41-56. https://doi.org/10.1111/hex.12619