Browse > Article
http://dx.doi.org/10.3961/jpmph.14.049

Assessing Reliability of Medical Record Reviews for the Detection of Hospital Adverse Events  

Ock, Minsu (Department of Preventive Medicine, University of Ulsan College of Medicine)
Lee, Sang-il (Department of Preventive Medicine, University of Ulsan College of Medicine)
Jo, Min-Woo (Department of Preventive Medicine, University of Ulsan College of Medicine)
Lee, Jin Yong (Public Health Medical Service, Seoul National University Boramae Medical Center)
Kim, Seon-Ha (Department of Nursing, Dankook University)
Publication Information
Journal of Preventive Medicine and Public Health / v.48, no.5, 2015 , pp. 239-248 More about this Journal
Abstract
Objectives: The purpose of this study was to assess the inter-rater reliability and intra-rater reliability of medical record review for the detection of hospital adverse events. Methods: We conducted two stages retrospective medical records review of a random sample of 96 patients from one acute-care general hospital. The first stage was an explicit patient record review by two nurses to detect the presence of 41 screening criteria (SC). The second stage was an implicit structured review by two physicians to identify the occurrence of adverse events from the positive cases on the SC. The inter-rater reliability of two nurses and that of two physicians were assessed. The intra-rater reliability was also evaluated by using test-retest method at approximately two weeks later. Results: In 84.2% of the patient medical records, the nurses agreed as to the necessity for the second stage review (kappa, 0.68; 95% confidence interval [CI], 0.54 to 0.83). In 93.0% of the patient medical records screened by nurses, the physicians agreed about the absence or presence of adverse events (kappa, 0.71; 95% CI, 0.44 to 0.97). When assessing intra-rater reliability, the kappa indices of two nurses were 0.54 (95% CI, 0.31 to 0.77) and 0.67 (95% CI, 0.47 to 0.87), whereas those of two physicians were 0.87 (95% CI, 0.62 to 1.00) and 0.37 (95% CI, -0.16 to 0.89). Conclusions: In this study, the medical record review for detecting adverse events showed intermediate to good level of inter-rater and intra-rater reliability. Well organized training program for reviewers and clearly defining SC are required to get more reliable results in the hospital adverse event study.
Keywords
Adverse event; Patient safety; Intra-rater reliability; Inter-rater reliability; Medical record review;
Citations & Related Records
연도 인용수 순위
  • Reference
1 World Health Organization, The research cycle: measuring harm [cited 2014 Sep 23]. Available from: http://www.who.int/patientsafety/research/strengthening_capacity/measuring_harm/en/.
2 Thomas EJ, Petersen LA. Measuring errors and adverse events in health care. J Gen Intern Med 2003;18(1):61-67.   DOI
3 World Health Organization. Assessing and tackling patient harm: a methodological guide for data-poor hospitals. Geneva: World Health Organization; 2010, p. 35-36.
4 Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991;324(6):370-376.   DOI
5 Rivard PE, Luther SL, Christiansen CL, Shibei Zhao, Loveland S, Elixhauser A, et al. Using patient safety indicators to estimate the impact of potential adverse events on outcomes. Med Care Res Rev 2008;65(1):67-87.   DOI
6 Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes BA, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991;324(6):377-384.   DOI
7 de Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA. The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 2008;17(3):216-223.   DOI
8 Vincent C. Patient safety. 2nd ed. Chichester: Wiley-Blackwell; 2010, p. 49-74.
9 Forster AJ, Taljaard M, Bennett C, van Walraven C. Reliability of the peer-review process for adverse event rating. PLoS One 2012;7(7):e41239.   DOI
10 Brennan TA, Localio RJ, Laird NL. Reliability and validity of judgments concerning adverse events suffered by hospitalized patients. Med Care 1989;27(12):1148-1158.   DOI
11 Lee SI, Kim Y, Lee JH, Lee JY, Jo MW, Ock M, et al. Korean protocol development and assessment to secure the safety of patients. Seoul: National Evidence-based Healthcare Collaborating Agency; 2012, p. 144-201 (Korean).
12 Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The quality in Australian health care study. Med J Aust 1995;163(9):458-471.
13 Zegers M, de Bruijne MC, Wagner C, Hoonhout LH, Waaijman R, Smits M, et al. Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Saf Health Care 2009;18(4):297-302.   DOI
14 Hulley SB, Cummings SR, Browner WS, Grady D, Newman TB. Designing clinical research. 4th ed. Philadelphia: Lippincott Williams & Wilkins; 2013, p. 32-42.
15 Hwang JI, Chin HJ, Chang YS. Characteristics associated with the occurrence of adverse events: a retrospective medical record review using the Global Trigger Tool in a fully digitalized tertiary teaching hospital in Korea. J Eval Clin Pract 2014;20(1):27-35.   DOI
16 Aranaz-Andres JM, Aibar-Remon C, Vitaller-Murillo J, Ruiz-Lopez P, Limen-Ramirez R, Terol-Garcia E, et al. Incidence of adverse events related to health care in Spain: results of the Spanish National Study of Adverse Events. J Epidemiol Community Health 2008;62(12):1022-1029.   DOI
17 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159-174.   DOI
18 Mendes W, Martins M, Rozenfeld S, Travassos C. The assessment of adverse events in hospitals in Brazil. Int J Qual Health Care 2009;21(4):279-284.   DOI
19 Soop M, Fryksmark U, Koster M, Haglund B. The incidence of adverse events in Swedish hospitals: a retrospective medical record review study. Int J Qual Health Care 2009;21(4):285-291.   DOI
20 Naessens JM, O’Byrne TJ, Johnson MG, Vansuch MB, McGlone CM, Huddleston JM. Measuring hospital adverse events: assessing inter-rater reliability and trigger performance of the Global Trigger Tool. Int J Qual Health Care 2010;22(4):266-274.   DOI
21 Davis P, Lay-Yee R, Briant R, Ali W, Scott A, Schug S. Adverse events in New Zealand public hospitals II: preventability and clinical context. N Z Med J 2003;116(1183):U624.