• Physical Therapy Korea
• /
• v.26 no.1
• /
• pp.8-18
• /
• 2019

Background: Numerous studies have used smartphone applications to measure the range of motion in different joints. In addition, studies measuring the active range of motion (AROM) of the craniocervical joint have revealed high reliability. However, the subjects in these studies were all healthy subjects. No study has yet been conducted to measure the inter-rater reliability for the AROM of the craniocervical joint in stroke patients. Objects: The purpose of this study was to investigate the inter-rater reliability of the AROM of the craniocervical joint using a smartphone. Methods: The participants included 21 subjects who had strokes (17 males and 4 females). Two raters evaluated six types of craniocervical AROM, including flexion, extension, lateral flexion to the hemiplegic side, lateral flexion to the non-hemiplegic side, rotation to the hemiplegic side, and rotation to the non-hemiplegic side, using a goniometer and a smartphone to investigate inter-rater reliability. The inter-rater reliability was analyzed by intraclass correlation coefficients (ICC). Results: The inter-rater reliability of the smartphone was good for extension, lateral flexion to the hemiplegic side, lateral flexion to the non-hemiplegic side, and rotation to the hemiplegic side [ICC(2,k)=.86~.88] and excellent for flexion [ICC(2,k)=.95]. The inter-rater reliability for rotation to the non-hemiplegic side was moderate [ICC(2,k)=.72]. Conclusion: These results suggest that the smartphone offers high inter-rater reliability for measurements of the craniocervical AROM in patients with stroke.

• The Journal of Korean Physical Therapy
• /
• v.22 no.4
• /
• pp.43-48
• /
• 2010

Purpose: This study was designed to determine the test-retest, inter-rater, and intra-rater reliability of the Pediatric Balance Scale (PBS) when applied to children with cerebral palsy (CP). Methods: Subjects were out-patient children with cerebral palsy at four CP clinics in Gyeonggyi-do and Chungcheong nam-do. For test-retest and inter-rater reliability studies, the PBS was used twice on 7 separate days by twenty-four children with CP. To assess intra-rater reliability, 10 CP subjects were selected by random sampling. Four pediatric-trained physical therapists with 2-13 years of clinical experience scored the children’s performance blindly, while replaying videotaped data. Results: There was no significant difference in total scores (ICC 3,1=0.89, 0.93, 0.90, and 0.91) measured by each of the four therapists on two occasions. The Inter-rater reliability assessed the 1st and 2nd time was high (1stICC 3,1=0.91, 2nd ICC 3,1=0.93). The intra-rater reliability measured by each of the four therapists using the 2nd scores was also high (ICC 3,1=0.98, 0.99, 0.97, and 0.98). Conclusion: The PBS is reliable. We believe that it can be used in characterizing children with CP.

• The Journal of Korean Physical Therapy
• /
• v.30 no.1
• /
• pp.29-34
• /
• 2018

Purpose: This study aimed to systematically analyze the reliability and validity of Korean versions of mobility and gait assessment tools for patients with stroke. Methods: Two reviewers conducted an independent literature search and systematically reviewed the literature. Literature published until June of 2017 was searched using search engines for electronic databases. The 2 reviewers reviewed the title and abstract of each article. Among the articles examined, we read the full text of those judged to be suitable for our study. We classified the selected data into research methods and results. The determination was made through mutual agreement. Results: The reviewers selected 5 articles related to the purpose of this study. The DGI was found to be moderately associated with BBS, 10MWT, and TUG in intra-rater reliability (ICC=0.92), inter-rater reliability (ICC=0.88), and concurrent validity tests. The FGA demonstrated a high level of reliability with intra-rater (ICC=0.92-0.95) and inter-rater reliability (ICC=0.91, 0.95). The intra-rater reliability of the RMI was high at ICC=0.98. Its concurrent validity showed a high association with STREAM and BI. The intra-rater reliability of the MAS was ICC=0.75-0.99. Its inter-rater reliability was very high, exceeding 0.99. The inter-rater reliability of the Tinetti-gait Scale was ICC=0.91, and its concurrent validity was moderately associated with the DGI, 10WMT, OLST, FM-L/E, and STS. Conclusion: The results of this study revealed that the reliability and validity of the Korean versions of the mobility and gait assessment tools were high.

• Journal of Preventive Medicine and Public Health
• /
• v.48 no.5
• /
• pp.239-248
• /
• 2015

Objectives: The purpose of this study was to assess the inter-rater reliability and intra-rater reliability of medical record review for the detection of hospital adverse events. Methods: We conducted two stages retrospective medical records review of a random sample of 96 patients from one acute-care general hospital. The first stage was an explicit patient record review by two nurses to detect the presence of 41 screening criteria (SC). The second stage was an implicit structured review by two physicians to identify the occurrence of adverse events from the positive cases on the SC. The inter-rater reliability of two nurses and that of two physicians were assessed. The intra-rater reliability was also evaluated by using test-retest method at approximately two weeks later. Results: In 84.2% of the patient medical records, the nurses agreed as to the necessity for the second stage review (kappa, 0.68; 95% confidence interval [CI], 0.54 to 0.83). In 93.0% of the patient medical records screened by nurses, the physicians agreed about the absence or presence of adverse events (kappa, 0.71; 95% CI, 0.44 to 0.97). When assessing intra-rater reliability, the kappa indices of two nurses were 0.54 (95% CI, 0.31 to 0.77) and 0.67 (95% CI, 0.47 to 0.87), whereas those of two physicians were 0.87 (95% CI, 0.62 to 1.00) and 0.37 (95% CI, -0.16 to 0.89). Conclusions: In this study, the medical record review for detecting adverse events showed intermediate to good level of inter-rater and intra-rater reliability. Well organized training program for reviewers and clearly defining SC are required to get more reliable results in the hospital adverse event study.

• The Journal of Korean Physical Therapy
• /
• v.22 no.6
• /
• pp.35-42
• /
• 2010

Purpose: To investigate the rater reliability of a Pediatric Balance Scale (PBS) for children with cerebral palsy, and to investigate possible differences among raters according to their clinical work experience and testing experience. Methods: Study participants included 18 children with spastic cerebral palsy who could walk. They were instructed by pediatric physical therapists, two of whom had ten years of clinical work experience and two who had less than one year of experience. The children's ability to achieve physical balance was videotaped for PBS items. The raters watched the tapes and evaluated each child twice. Rater reliability was analyzed using the intraclass correlation coefficient (ICC). Differences between experienced and novice raters were analyzed using a paired t-test. The statistical significance level was set to 0.05. Results: The total PBS scores averaged 45.78~48.00 and 45.72~47.67 for first and second tests. Intra-rater reliability was very high (ICC=0.89~0.99), and the repeated measurement coincidence was high (p>0.05). Inter-rater reliability was high (ICC=0.83~0.84), but there was a bit of a difference in the coincidence (p<0.05). The experienced raters' reliability and coincidence were higher than those of the novices, and there were differences in reliance and coincidence between experienced and novice raters (p<0.05). Conclusion: Inter-rater and intra-rater reliability is very high. However, rater reliability showed defferences depending on clinical work experience and testing experience. When testing pediatric patients with the PBS, the rater's clinical experience and test experience may affect the test results.

• Journal of the Korean Academy of Clinical Electrophysiology
• /
• v.10 no.1
• /
• pp.29-37
• /
• 2012

Purpose : The purpose of this study was to establish intra-rater, inter-rater, test-retest reliability, and concurrent validity of figure-of-8 walk test in people with stroke. Methods : The subjects of this study were 17 patients who were diagnosed with a stroke. Subjects were tested twice by the same raters, with 1 day between tests. Subjects were assessed by two physical therapists. Test-retest reliability was calculated using intraclass correlation coefficients (ICC). The concurrent validity was demonstrated by spearman correlation of F8WT with 10m walking test (10MWT), timed up and go test (TUG), Berg balance scale (BBS), dynamic gait index (DGI) and four square step test (FSST). Results : Intra-rater, inter-rater, test- retest of F8WT time, showed high reliability. Intra-rater, inter-rater, test-retest of F8WT steps demonstrated high reliability. Intra-rater, inter-rater, test-retest of F8WT total smoothness score showed below moderate reliability. There was a significant positive correlation of F8WT time with 10MWT, TUG, FSST. There was a significant negative correlation of F8WT time with DGI, BBS. There was a significant positive correlation of F8WT steps with 10MWT, TUG, FSST. There was a significant negative correlation of F8WT steps with DGI. There was a significant positive correlation of F8WT test total smoothness score with BBS. Conclusion : The time, and number of steps in F8WT show high inter, intra-rater, test-retest reliability. The F8WT smoothness shows below moderate reliability. The F8WT shows high concurrent validity with other comparable balance, and walking tests. The F8WT is a valid and reliable measure for assessing walking function in patients with a stroke.

• Physical Therapy Korea
• /
• v.21 no.1
• /
• pp.1-12
• /
• 2014

The purpose of this study were to determine the intra-rater and inter-rater reliability of shoulder passive range of motion measurement using the "Clinometer + bubble level", a smartphone application and to compare with the intra-rater and inter-rater reliability of measurement using a goniometer. Twenty six patients with stroke were recruited for this study. Two raters measured the passive range of motion of four types of shoulder movements (forward flexion; FF, abduction; ABD, external rotation at $90^{\circ}$ abduction; ER90 and internal rotation at $90^{\circ}$ abduction; IR90) using a goniometer and a smartphone to determine within-day inter-rater reliability. A retest session was performed thirty minutes later to determine within-day intra-rater reliability. The reliability was assessed using intraclass correlation coefficients (ICC) and the standard error of measurement (SEM). The ICC (2,1) for the inter-rater reliabilities of the goniometer and smartphone were good in FF and ABD [ICC (2,1)=.75~.87] and excellent in ER90 [ICC (2,1)=.90~.95]. The intra-rater reliabilities for the goniometer and smartphone were good or more than good, with an ICC (3,1) value >.75, the exception was IR90 measured by rater 2 on the smartphone. These results suggest that smartphone could be used as an alternative method tool for measurement of passive shoulder range of motion in patients with stroke.

• The Journal of Internal Korean Medicine
• /
• v.34 no.3
• /
• pp.231-239
• /
• 2013

Objectives : This study was performed to evaluate inter-rater and intra-rater reliability of interpretation and reproducibility of a pulse analyser (MAXMAC27-Plus). Methods : 38 of 40 volunteers completed the pulse analysis consecutively. Three Korean medical doctors who had at least 2 years of clinical experience interpreted the pulse waves for 3 aspects of size, depth and shape, then inter-rater reliability and crude agreement was obtained. Reinterpretation was done 2 weeks later and intra-rater reliability and crude agreement was obtained. Intra-rater reliability and crude agreement between 1st and 2nd measurement was calculated. Cohen's weighted kappa for size, Cohen's kappa for depth and shape were used as statistical analysis. Results : Inter-rater reliability of size, depth and shape among 3 raters was 0.598, 0.604, and 0.312, respectively, showing moderate to substantial agreement. Average intra-rater reliability between 1st and 2nd interpretation of size, depth and shape was 0.806, 0.705, and 0.638, respectively, showing substantial to almost perfect agreement. However, intra-rater reliability between consecutive measurements of size, depth and shape was 0.221, 0.121, and 0.194, respectively, which showed only poor to fair agreement. Conclusions : Intra-rater and inter-rater reliability of one pulse wave showed relatively high concordance. Training by a clinical expert may effect better concordance among raters. Test-retest reliability showed poor agreement. Improvement of measurement technique and device performance will be needed.

• The Journal of the Korean dental association
• /
• v.58 no.5
• /
• pp.284-291
• /
• 2020

This study analyzed the change in the rater reliability based on the student's practice evaluation process conducted at Yonsei University College of Dentistry. Through this, we suggest the significance of the rater calibration training in the student's practical evaluation of dental college. Nine professors from the department of Conservative Dentistry, Yonsei University College of Dentistry, analyzed the results of class II restoration cases twice in 2017 and once in 2018. Intra Class Correlation (ICC) which is a statistic used to determine the consistency of raters with three or more scores, was also calculated. ICC values increased as raters participated in rater calibration meetings and grading experiences. This shows that the rater reliability is related to the grading experience and feedback from calibration meeting. Based on the results of previous studies that grading experiences and rater calibration training can cause a meaningful change in rater behavior, we propose to conduct rater calibration training to ensure the evaluator reliability.

• Journal of East-West Nursing Research
• /
• v.26 no.1
• /
• pp.91-99
• /
• 2020

Purpose: The study aims to assess the inter-rater reliability of the Korean Triage and Acuity Scale between the research nurses and the triage nurses. Methods: Interrater reliability was measured on 400 adult (≧15) and 400 pediatric (<15) patients who visited the emergency medical center from January 4 to June 30, 2018. Results: The study result showed that the inter-rater reliability of the Korean Triage and Acuity Scale was substantial, with κ=.73 (95% Confidence interval= .68-.78) and 77.0 percent agreement. The inter-rater of Pediatric Korean Triage and Acuity Scale was also substantial, with κ=.76 (95% Confidence interval= .71-.82) and 83.8 percent agreement. Conclusion: Although the inter-rater reliability of the Korean Triage and Acuity Scale was acceptable, the percent agreement was lower than the desirable level (<80.0%). It was confirmed that Pediatric Korean Triage and Acuity Scale had an acceptable level of inter-rater reliability and percent agreement for clinical use. Efforts should be made to improve the reliability in the future.