• The Journal of the Korea Contents Association
• /
• v.21 no.3
• /
• pp.410-418
• /
• 2021

In a program that runs an engineering education certification, program outcomes refer to the knowledge, skills, and attitudes a student must have until graduation. In general, capstone design is used as a tool for evaluating program outcomes. This paper applies the intraclass correlation coefficient (ICC) to measure the raters' reliability in assessing program outcomes. Several raters evaluate program outcomes, and the result is used to obtain the raters' ICC. ICC measures the reliability of ratings or measurements for clusters - data that has been collected as groups or sorted into groups. If the ICC is close to 1, it means that the reliability among the raters is high. We evaluated the proposed method's usefulness through case analysis. As a method for assessing an evaluation tool's objectivity, multiple raters measure the same evaluation tool. As a result, we measured the ICC values for all POs, and analyzed the cause for the low measured POs. We applied this method to evaluate program outcomes of the Department of Computer Engineering in the past two years. As a result, we derived guidelines for improvement and program outcomes.

• International Journal of Internet, Broadcasting and Communication
• /
• v.10 no.3
• /
• pp.65-72
• /
• 2018

The purpose of this study was to assess the intra-tester and inter-tester reliability and validity of elbow flexion range of motion using a smart phone used by the general public. Thirty-one general people participated in this study. The range of motion in active elbow flexion was measured with a goniometer and smart phone over two times by two observers. The intra-taster and inter-tester reliability were evaluated using the intraclass correlation coefficient(ICC). The validity was measured by Pearson's correlation coefficient. The intra-observer reliability was good in all measured items ICC>0.900. The inter-observer reliability was high with ICC>0.806. All correlation coefficients of android app and goniometer were greater than 0.868 and showed a significant positive correlation (p<0.01). The range of motion measurement with a smart phone showed acceptable reliability. Therefore, using a smartphone to measure the range of motion of the elbow joint could provide convenience and economic benefits.

• The Journal of Korean Academy of Sensory Integration
• /
• v.20 no.2
• /
• pp.11-21
• /
• 2022

Objective : This study examined the test-retest reliability and interrater reliability of the Sensory Processing Scale for Children (SPS-C). Method : Senventy primary caregivers of children with sensory processing difficulties and 3 years old participated in the study. The subjects were recruited through child development centers, welfare centers, and acquaintances located in Seoul, Gyeonggi-do, Busan, and Gyeongsang-do. The test-retest reliability verification targeted 20 main caregivers of children with difficulty in sensory processing. Re-evaluation was performed within 7 to 14 days after the initial evaluation, and Pearson's correlation coefficient was used to confirm the relevance between the two time points, and the Intraclass correlation coefficient was used to confirm the degree of agreement. The interrater reliability verification was conducted with 18 primary caregivers and 18 subsidiary caregivers of children with sensory processing difficulties. Each caregiver evaluated the same child, and the Intraclass correlation coefficient was used to confirm the agreement between the two sets of caregivers. Results : The test-retest reliability was Pearson's correlation coefficient r=.914 and intraclass correlation coefficient ICC=.939, indicating a high level of relevance and agreement. The interrater reliability was an Intraclass correlation coefficient ICC=.727, which showed a moderate level of agreement, but the tactile area (ICC=.455) and proprioceptive area (ICC=.439) were not statistically significant and showed a low degree of agreement. Conclusion : Through this study, it was confirmed that the children's Sensory Processing Scale for Children (SPS-C) is a stable evaluation tool with test-retest reliability and interrater reliability verified, and it will be able to provide help in standardization studies for future clinical use.

• Journal of Yeungnam Medical Science
• /
• v.35 no.1
• /
• pp.63-69
• /
• 2018

Background: Kinovea software that tracking semi-automatically the motion in video screen has been used to study motion-related tasks in several studies. However, the validation of this software in open field test to assess locomotor activity have not been studied yet. Therefore, this study aimed to examine the reliability and validity of this software in analyzing locomotor activities. Methods: Thirty male Institute Cancer Research mice were subjected in this study. The results examined by this software and the classical method were compared. Test-retest reliability and inter-rater reliability were analyzed with Pearson's correlation coefficient and intraclass correlation coefficient (ICC). The validity of this software was analyzed with Pearson's correlation coefficient. Results: This software showed good test-retest reliability (ICC=0.997, 95% confidence interval [CI]=0.975-0.994, p<0.001). This software also showed good inter-rater reliability (ICC=0.987, 95% CI=0.973-0.994, p<0.001). Furthermore, in three analyses for the validity of this software, there were significant correlations between two methods (Pearson's correlation coefficient=0.928-0.972, p<0.001). In addition, this software showed good reliability and validity in the analysis locomotor activity according to time interval. Conclusion: This study showed that this software in analyzing drug-induced locomotor activity has good reliability and validity. This software can be effectively used in animal study using the analysis of locomotor activity.

• The Korean Journal of Applied Statistics
• /
• v.23 no.3
• /
• pp.497-510
• /
• 2010

In medical practice and research, the problem of assessing reliability between two or more quantitative measures is quite common. Intraclass correlation coefficient(ICC) is commonly used to scale of reliability. Some methods were developed to calculate the required number of subjects, raters or replicates in one-way or two-way random ANOVA models. This paper, studies and compares the performance of four methods such as Walter et al. (1998), Giraudeau and Mary (2001), Saito et al. (2006) and Bonett (2002). In order to compare the efficiency of methods we compare the number of subjects, replicates and the width of confidence interval of ICC needed for some specific ICC values. In the case of subject size, Giraudeau's method is the best. In case of the number of replicates, Saito's method was superior to others. The width of confidence interval of ICC was narrower for Giraudeau's method than any others.

• The Korean Journal of Applied Statistics
• /
• v.27 no.1
• /
• pp.89-102
• /
• 2014

In this study, we analyze the effect on the smoking status in the Seoul Metropolitan area using a multilevel logistic model with Community Health Survey data from the Korea Centers for Disease Control and Prevention. Intraclass correlation coefficient (ICC), profiling analysis and two types of predicted value were used to determine the appropriate multilevel analysis level. Sensitivity, specificity, percentage of correctly classified observations (PCC) and ROC curve evaluated model performance. We showed the applicability for multilevel analysis allowed for the possibility that different factors contribute to within group and between group variability using survey data.

• Journal of the Korean Society of Physical Medicine
• /
• v.10 no.2
• /
• pp.1-7
• /
• 2015

PURPOSE: The aims of this study were to assess the intra-tester and inter-tester reliability and validity of hip range of motion using a smart phone in hip operative patients. METHODS: Twenty-five patients (eight total hip arthroplasty and seventeen femur neck fracture) participated in this study. The range of motions in active and passive hip flexion, abduction and external rotation were measured with a goniometer and smart phone over two times by two observers. The intra-tester and inter-tester reliability were evaluated using the intraclass correlation coefficient (ICC2,1). The validity was measured by Pearson's correlation coefficient RESULTS: The intra-observer reliability was good in all measured items (ICC>0.78). The inter-observer reliability was high with ICC (>0.90). All correlation coefficients of smart phone and goniometer was greater than 0.85 and showed a significant positive correlation (p<0.01). CONCLUSION: The range of motion measurement with a smart phone showed acceptable reliability. Therefore, it could be convenient and have economical benefits to measure the range of motion of the hip joint using a smart phone.

• The Journal of Korean Academy of Sensory Integration
• /
• v.14 no.1
• /
• pp.1-8
• /
• 2016

Objective : The purpose of this study was to determine the reliability of Test of Playfulness (ToP) in Attention Deficit Hyperactivity Disorder (ADHD) children. Methods : Twenty ADHD children participated in this study. To determine inter-rater reliability, two raters was evaluated by Test of Playfulness scores and analyzed Kappa coefficient and total agreement and intraclass correlation coefficient (ICC). Results : Total agreement was the range of 75%~95% and the Kappa coefficients was the range of. 45~1.00. Analyzing inter-rater reliability by ICC showed very high reliability: Intrinsic motivation was .96, internal control was .98, suspension of reality was .90, framing was .98, total score was .99. Conclusion : A reliability of Test of Playflness of children with ADHD showed that high. On the basis of this study, it will provide availability for clinical evaluation of children with ADHD, and basic study on the play of children with ADHD.

• Journal of the Korean Society of Physical Medicine
• /
• v.10 no.3
• /
• pp.47-55
• /
• 2015

PURPOSE: Stoke is one of most common disabling conditions and it is still lacking of measuring patient's functioning level. The aim of the study was to develop Korean language version of stroke impact scale 3.0. METHODS: Korean version of stroke impact scale 3.0 was developed in idiomatic modern Korean with a standard protocol of multiple forward and backward translations and an expert reviews to achieve equivalence with the original English version. Interviews with clinicians who were currently managing patients with stroke were also conducted for language evaluation. A reliability test was performed to make final adaptation using a pre-final version. To assess the reliability of the translated questionnaire, the intraclass correlation coefficient (ICC) was calculated for each domain of the scale. RESULTS: Thirty subjects (16 male, 14 female) aged from 20 to 75 years old participated to review the translated questionnaire. Reliability of each domain of the questionnaire was found to be good in strength (ICC=0.74), ADL (ICC=0.81), mobility (ICC=0.90), hand function (ICC=0.80) and social participation (ICC=0.79), communication (ICC=0.77) with total (ICC=0.76). However, domains of memory and thinking (ICC=0.66), and emotion (ICC=0.27) and showed poor reliability. CONCLUSION: This study indicates that the Korean version of SIS 3.0 was successfully developed. Future study needed for obtaining the validity of the Korean version of SIS 3.0.

• Journal of Trauma and Injury
• /
• v.36 no.1
• /
• pp.8-14
• /
• 2023

Purpose: The aim of this study was to assess the agreement between intraoperative transcranial sonography (TCS) and postoperative computed tomography (CT) in patients with traumatic brain injuries. Methods: We performed a retrospective cross-sectional study of 35 patients who underwent TCS during surgery, among those who presented to a regional trauma center and underwent decompressive craniectomy between January 1, 2017 and April 30, 2020. Results: The mean difference between TCS and CT in measuring the midline shift was -1.33 mm (95% confidence interval, -2.00 to -0.65; intraclass correlation coefficient [ICC], 0.96; P<0.001). An excellent correlation was found between TCS and CT in assessing contralateral subdural hematomas (ICC, 0.96; P<0.001) and focal hematoma lesions (ICC, 0.99; P<0.001). A very good correlation between TCS and CT was found for measurements of ventricle width (ICC, 0.92; P<0.001). Conclusions: TCS during surgery is considered an effective diagnostic tool for the detection of intraoperative parenchymal changes in patients with traumatic brain injuries.