Physical Therapy Korea (한국전문물리치료학회지)

Korean Research Society of Physical Therapy (한국전문물리치료학회)
권호 표지
DOI QR코드

DOI QR Code

Concordance Rate Between the Ratings of Clinician and Self Ratings of Worker on a Functional Capacity Evaluation

  • Choi, Bong-sam (Dept. of Physical Therapy, College of Health and Welfare, Woosong University, Advanced Institute of Convergence Sport Rehabilitation, Woosong University)
  • Received : 2016.10.07
  • Accepted : 2016.11.07
  • Published : 2016.11.19
Download PDF
⟨ Previous Next ⟩

Abstract

Background: Functional capacity evaluations (FCEs) are designed to systematically assess the capacity to perform work-related tasks and to determine worker's ability to return to the previous job following work-related injuries. These evaluations may be rated either by clinician or worker. There has been a lack of consensus between the two scoring methods. Objects: This study aimed: 1) to confirm if the data are fit to the Rasch rating scale model and 2) to investigate the item-level concordance rate between the ratings of clinician and injured worker of the FCE. Methods: A cross-sectional study was conducted with a sample (n=124) of a rehabilitation program with the Occupational Rehabilitation Data Base for workers with low back pain. The functional capacity evaluation at admission and discharge was administered to clinicians and workers. The data were analyzed using both classical test theory-based Pearson's r and intra-class coefficient followed by item-level analysis with Rasch rating scale model. Results: All items of the FCE, except sitting items rated by clinician at admission and handling items rated by both clinician and worker throughout admission and discharge, were acceptable fit statistics with minor out of ranges for a misfit criterion. This may indicate that the items of the FCE overall fit to the Rasch rating scale model. Few problematic items responding differently to clinician and worker both at admission and discharge were detected with the differential item functioning analysis despite the excellent concordance rate using the two conventional statistics-sitting and handling items at admission and handling item at discharge. Conclusion: The item-level speculations using Rasch analysis of the FCE demonstrate that the ratings of clinician and self ratings of worker were psychometrically acceptable though there was an apparent discrepancy between the raters both at admission and discharge.

Keywords

References

  1. Beninato M, Portney LG, Sullivan PE. Using the International Classification of Functioning, Disability and Health as a framework to examine the association between falls and clinical assessment tools in people with stroke. Phys Ther. 2009; 89(8):816-825. https://doi.org/10.2522/ptj.20080160
  2. Bond TG, Fox CM. Applying the Rasch Model: Fundamental measurement in the human sciences. 2nd ed. Mahwah, NJ, Lawrence Erlbaum Associates Publishers, 2001:23-28.
  3. Bovend'Eerdt TJ, Dawes H, Izadi H et al. Agreement between two different scoring procedures for goal attainment scaling is low. J Rehabil Med. 2011; 43(1):46-49. https://doi.org/10.2340/16501977-0624
  4. Bray P, Bundy AC, Ryan MM, et al. Health-related quality of life in boys with Duchenne muscular dystrophy: Agreement between parents and their sons. J Child Neurol. 2010;25(10):1188-1194. https://doi.org/10.1177/0883073809357624
  5. Chen JJ. Functional capacity evaluation & disability. Iowa Orthop J. 2007;27:121-127.
  6. Choi BS, Park SY. Responsiveness comparisons of self-report versus therapist-scored functional capacity for workers with low back pain. Phys Ther Korea. 2012;19(3):91-97. https://doi.org/10.12674/ptk.2012.19.3.091
  7. Cutler RB, Fishbain DA, Steele-Rosomoff R, et al. Relationships between functional capacity measures and baseline psychological measures in chronic pain patients. J Occup Rehabil. 2003;13(4):249-258. https://doi.org/10.1023/A:1026220704974
  8. Davis E, Nicolas C, Waters E, et al. Parent-proxy and child self-reported health-related quality of life: Using qualitative methods to explain the discordance. Qual Life Res. 2007;16(5):863-871. https://doi.org/10.1007/s11136-007-9187-3
  9. De Civita M, Regier D, Alamgir AH, et al. Evaluating health-related quality-of-life studies in paediatric populations: Some conceptual, methodological and developmental considerations and recent applications. Pharmacoeconomics. 2005;23(7):659-685. https://doi.org/10.2165/00019053-200523070-00003
  10. Eiser C, Morse R. Can parents rate their child's health-related quality of life? Results of a systematic review. Qual Life Res. 2001;10(4):347-357. https://doi.org/10.1023/A:1012253723272
  11. Fishbain DA, Abdel-Moty E, Cutler R, et al. Measuring residual functional capacity in chronic low back pain patients based on the Dictionary of Occupational Titles. Spine (Phila Pa 1976). 1994;19(8):872-880. https://doi.org/10.1097/00007632-199404150-00002
  12. Fishbain DA, Cutler RB, Rosomoff H, et al. Validity of the dictionary of occupational titles residual functional capacity battery. Clin J Pain. 1999; 15(2):102-110. https://doi.org/10.1097/00002508-199906000-00006
  13. Gross DP. Measurement properties of performance-based assessment of functional capacity. J Occup Rehabil. 2004;14(3):165-174. https://doi.org/10.1023/B:JOOR.0000022759.30446.4f
  14. Gross DP, Battie MC. The prognostic value of functional capacity evaluation in patients with chronic low back pain: Part 2: Sustained recovery. Spine (Phila Pa 1976). 2004;29(8):920-924. https://doi.org/10.1097/00007632-200404150-00020
  15. Haley SM, Coster WJ, Andres PL, et al. Activity outcome measurement for postacute care. Med Care. 2004;42(1 Suppl):I49-I61.
  16. Huang HY. Effects of the common scale setting in the assessment of differential item functioning. Psychol Rep. 2014;114(1):104-125. https://doi.org/10.2466/03.PR0.114k11w0
  17. James CL, Reneman MF, Gross DP. Functional capacity evaluation research: Report from the second international functional capacity evaluation research meeting. J Occup Rehabil. 2016;26(1): 80-83. https://doi.org/10.1007/s10926-015-9589-y
  18. Kuijer PP, Gouttebarge V, Brouwer S, et al. Are performance-based measures predictive of work participation in patients with musculoskeletal disorders? A systematic review. Int Arch Occup Environ Health. 2012;85(2):109-123. https://doi.org/10.1007/s00420-011-0659-y
  19. Lim Y, Velozo CA, Bendixen RM. The level of agreement between child self-reports and parent proxy-reports of health-related quality of life in boys with Duchenne muscular dystrophy. Qual Life Res. 2014;23(7):1945-1952. https://doi.org/10.1007/s11136-014-0642-7
  20. Linacre JM. Optimizing rating scale category effectiveness. J Appl Meas 2002;3(1):85-106.
  21. Linacre JM. Rasch model estimation: Further topics. J Appl Meas. 2004;5(1):95-110.
  22. Magaziner J, Zimmerman SI, Gruber-Baldini AL, et al. Proxy reporting in five areas of functional status. Comparison with self-reports and observations of performance. Am J Epidemiol. 1997;146(5):418-428. https://doi.org/10.1093/oxfordjournals.aje.a009295
  23. McHugh ML. Interrater reliability: The kappa statistic. Biochem Med (Zagreb). 2012;22(3):276-282.
  24. Ratzon NZ, Amit Y, Friedman S, et al. Functional capacity evaluation: Does it change the determination of the degree of work disability? Disabil Health J. 2015;8(1):80-85. https://doi.org/10.1016/j.dhjo.2014.08.004
  25. Rouquette A, Cote SM, Hardouin JB, et al. Rasch modelling to deal with changes in the questionnaires used during long-term follow-up of cohort studies: A simulation study. BMC Med Res Methodol. 2016;16(1):105. https://doi.org/10.1186/s12874-016-0211-6
  26. Soer R, Groothoff JW, Geertzen JH, et al. Pain response of healthy workers following a functional capacity evaluation and implications for clinical interpretation. J Occup Rehabil. 2008;18(3):290-298. https://doi.org/10.1007/s10926-008-9132-5
  27. Taherbhai HM, Young MJ. Pre-equating: A simulation study based on a large scale assessment model. J Appl Meas. 2004;5(3):301-318.
  28. Teresi JA. Statistical methods for examination of differential item functioning (DIF) with applications to cross-cultural measurement of functional, physical and mental health. J Ment Health Aging. 2001;7(1):31-40.
  29. Trippolini MA, Dijkstra PU, Geertzen JH, et al. Construct validity of functional capacity evaluation in patients with whiplash-associated disorders. J Occup Rehabil. 2015;25(3):481-492. https://doi.org/10.1007/s10926-014-9555-0
  30. United States Department of Labor. Dictionary of Occupational Titles. 4th eds. Supplement. Washington, DC, United States Government Printing, 1986:189-192.
  31. van der Linden FA, Kragt JJ, Hobart JC, et al. Proxy measurements in multiple sclerosis: Agreement between patients and their partners on the impact of multiple sclerosis in daily life. J Neurol Neurosurg Psychiatry. 2006;77(10):1157-1162. https://doi.org/10.1136/jnnp.2006.090795
  32. Velozo CA, Choi B, Zylstra SE, et al. Measurement qualities of a self-report and therapist-scored functional capacity instrument based on the Dictionary of Occupational Titles. J Occup Rehabil. 2006;16(1):109-122.
  33. Velozo CA, Kielhofner G, Lai JS. The use of Rasch analysis to produce scale-free measurement of functional ability. Am J Occup Ther. 1999;53(1):83-90. https://doi.org/10.5014/ajot.53.1.83
  34. Velozo CA, Santopoalo R. Training Manual: Occupational rehabilitation data base manual. Chicago, University of Illinois at Chicago, 1994:12-25.
  35. Viera AJ, Garrett JM. Understanding interobserver agreement: The kappa statistic. Fam Med. 2005;37(5):360-363.
  36. White-Koning M, Arnaud C, Dickinson HO, et al. Determinants of child-parent agreement in quality-of-life reports: A European study of children with cerebral palsy. Pediatrics. 2007;120(4): e804-e814. https://doi.org/10.1542/peds.2006-3272
  37. Wright BD, Linacre JM. Observations are always ordinal; measurements, however, must be interval. Arch Phys Med Rehabil. 1989;70(12):857-860.
  38. Wright BD, Stone MH. Best Test Design. Rasch Measurement. Chicago, MESA Press, 1979:93-95.