• Journal of Korean Neurosurgical Society
• /
• v.59 no.6
• /
• pp.577-583
• /
• 2016

Objective : The inter-rater reliability of the modified Knosp's classification was measured before the analysis. The clinical validity of the parasellar extension grading system was evaluated by investigating the extents of resection and complication rates among the grades in the endoscopic endonasal transsphenoidal surgery (EETS) for pituitary adenomas. Methods : From November 2008 to August 2015, of the 286 patients who underwent EETS by the senior author, 208 were pituitary adenoma cases (146 non-functioning pituitary adenomas, 10 adrenocorticotropic hormone-secreting adenomas, 31 growth hormone-secreting adenomas, 17 prolactin-secreting adenomas, and 4 thyroid-stimulating hormone-secreting adenomas; 23 microadenomas, 174 macroadenomas, and 11 giant adenomas). Two neurosurgeons and a neuroradiologist independently measured the degree of parasellar extension on the preoperative sellar MRI according to the modified Knosp's classification. Inter-rater reliability was statistically assessed by measuring the intraclass correlation coefficient. The extents of resection were evaluated by comparison of the pre- and post-operative MR images; the neurovascular complications were assessed by reviewing the patients' medical records. The extent of resection was measured in each parasellar extension grade; thereafter, their statistical differences were calculated. Results : The intraclass correlation coefficient value of reliability across the three raters amounted to 0.862. The gross total removal (GTR) rates achieved in each grade were 70.0, 69.8, 62.9, 21.4, 37.5, and 4.3% in Grades 0, 1, 2, 3A, 3B, and 4, respectively. A significant difference in the extent of resection was observed only between Grades 2 and 3A. In addition, significantly higher complication rates were observed in the groups above Grade 3A. Conclusion : Although the modified Knosp's classification system appears to be complex, its inter-rater reliability proves to be excellent. Regarding the clinical validity of the parasellar extension grading system, Grades 3A, 3B, and 4 have a negative predictive value for the GTR rate, with higher complication rates.

• The Journal of Korean Academy of Sensory Integration
• /
• v.17 no.3
• /
• pp.14-25
• /
• 2019

Objective : This study aims to verify the content validity and inter-rater reliability of the Korean version of the Peabody Developmental Motor Scale 2 (PDMS-2) and to identify the concurrent validity by comparing it with the Korean version of the Bruininks-Oseretsky Test of Motor Proficiency-2 (BOT-2). Methods : PDMS-2 was translated by the researcher and an eighth-year clinical occupational therapist. The content consistency of the Korean version of the PDMS-2 was verified by three professors with experience using it. After the verification of the content consistency of the PDMS-2 by the five clinical occupational therapists and the additional revision, the Korean version of the PDMS-2 was completed. The researcher and another occupational therapist evaluated the Korean version of PDMS-2 in 50 children and measured the inter-rater reliability. Concurrent validity was measured by comparing the results of the Korean version of PDMS-2 and Korean version of BOT-2. Results : The content consistency test showed overall agreement of mean 3.45, and the content understanding test showed a high level of understanding of mean 3.69. The inter-rater reliability and concurrent validity of the Korean version of the PDMS-2 showed a statistically significant correlation. Conclusion : The Korean version of the PDMS-2 showed high content understanding, reliability, and validity. It can assist clinicians and researchers who work in fields related to child treatment or development.

• Physical Therapy Rehabilitation Science
• /
• v.10 no.3
• /
• pp.379-386
• /
• 2021

Objective: The purpose of this study was to systematically review the trunk impairment scale that are used to assess the trunk control of stroke patients. Design: A systematic review Methods: Stroke subjects were categorized as acute, subacute, chronic. In this systematic review, the studies published between 2000 and 2020 were selected. A literature search using the keywords 'QUADAS', 'stroke', 'trunk impairment scale'. Data sources included RISS, GOOGLE Scholar and DBpia. We assessed the quality of assessment tools using Quality Assessment of Diagnostic Accuracy Studies tool. Results: We reviewed 18 studies. 7 of the 18 studies reported reliability results, 10 reported validity results. The QUADAS tool quality evaluation of 17 literatures extracted except for one randomized control test among 18 literatures showed a range of 3 to 13 points. 5 of the 18 studies are presented with the Cronbach alpha coefficient indicating reliability using internal consistency, all of which are more than 0.8. All studies that presented test-retest reliability, intra-rater reliability, and inter-rater reliability showed high agreement with an intra-class correlation coefficient of 0.75 or more. Conclusions: A systematic review of the study of the application of the trunk impairment scale for stroke patients will help provide criteria for future studies and application of the trunk impairment scale in clinical practice.

• The Journal of Korean Association of Computer Education
• /
• v.17 no.2
• /
• pp.39-51
• /
• 2014

In high-stakes tests for large groups, the efficiency with which students' responses are distributed to raters and how systematic scoring procedures are managed is important to the overall success of the testing program. In the scoring of constructed response items, it is important to understand whether the raters themselves are making consistent judgments on the responses, and whether these judgments are similar across all raters in order to establish measures of rater reliability. The purpose of this study was to design, develop and carry out a pilot test of an online scoring system for constructed response items administered in a paper-and-pencil test to large groups, and to verify the system's reliability. In this study, we show that this online system provided information on the scoring process of individual raters, including intra-rater and inter-rater consistency, compared to conventional scoring methods. We found this system to be especially effective for obtaining reliable and valid scores for constructed response items.

• Korean Journal of Adult Nursing
• /
• v.25 no.1
• /
• pp.24-32
• /
• 2013

Purpose: The aim of the study was to identify the most appropriate fall-risk assessment tool for neurological patients in an acute care setting. Methods: This descriptive study compared the reliability and validity of three fall-risk assessment tools (Morse Fall Scale, MFS; St Thomas's Risk Assessment Tool in Falling Elderly Inpatients, STRATIFY; Hendrich II Fall Risk Model, HFRM II). We assessed patients who were admitted to the Department of Neurology, Neurosurgery, and Rehabilitation at Asan Medical Center between July 1 and October 31, 2011, using a constructive questionnaire including general and clinical characteristics, and each item from the three tools. We analyzed inter-rater reliability with the kappa value, and the sensitivity, specificity, predictive value, and the area under the curve (AUC) of the three tools. Results: The analysis included 1,026 patients, and 32 falls occurred during this study. Inter-rater reliability was above 80% in all three tools. and the sensitivity was 50.0% (MFS), 84.4%(STRATIFY), and 59.4%(HFRM II). The AUC of the STRATIFY was 82.8. However, when the cutoff point was regulated as not 50 but 40 points, the AUC of the MFS was higher at 83.7. Conclusion: These results suggest that the STRATIFY may be the best tool for predicting falls for acute neurological patients.

• The Journal of Korean Medicine
• /
• v.38 no.3
• /
• pp.111-123
• /
• 2017

Objectives: To develop and investigate the reliability of the pathologic aging scale based on korean medical theory and korean medical pattern identification for dementia. Methods: We searched the textbook of korean neurophychiatry and Donguibogam and selected items through professional consensus. We compared between dementia(n=40) and normal elderly(n=38) and tested the reliability of two scales. Results: After professional consensus, we drafted the Korean Medical Pathologic Aging Scale(12 items, Likert 3 scale) and Korean Medical Pattern Identification for Dementia(4 patterns, 28 items, Likert 5 scale). On Korean Medical Pathologic Aging Scale, There is no significant difference between two groups. We had good internal consistency(Cronbach's alpha = 0.6) and test-retest reliability(r=0.631) but low inter-rater reliability(r=0.430). On Korean Medical Pattern Identification for Dementia, dementia patients diagnosed with Qi deficiency are significantly more than those in normal group. We had fairly good internal consistency(Cronbach's alpha = 0.574) and excellent test-retest(kappa= .800) and inter-rater reliability(kappa = .733). Conclusions: Korean Medical Pattern Identification for Dementia is appropriate for diagnosing korean medical pattern. But Korean Medical Pathologic Aging Scale isn't appropriate to discriminate dementia from normal elderly because of many subjective items. Therefore objective measurement of sensory dysfunction would be needed to measure pathologic aging based on korean medical theory.

• Journal of Preventive Medicine and Public Health
• /
• v.55 no.4
• /
• pp.334-341
• /
• 2022

Objectives: This study was conducted to develop the Mobility to Participation Assessment Scale for Stroke (MPASS) and assess its content validity, internal consistency, inter-rater and intra-rater reliability, and convergent validity in people with stroke living in the community. Methods: The MPASS was developed using published data on mobility-related activity and participation timing in elderly individuals, and then reviewed by community physical therapists. Content validity was established by reaching a consensus of experienced physical therapists in a focus group. The MPASS was scored for 32 participants with stroke (mean age 61.75±4.92 years) by 3 individual testers. Reliability was examined using the intraclass correlation coefficient (ICC), internal consistency using the Cronbach alpha coefficient (α), and convergent validity using the Pearson correlation coefficient (r) to compare the MPASS to the Modified Rivermead Mobility Index as a referent test of mobility. Results: The MPASS consists of 8 items, and its scoring system provides information on the ability of people with stroke to reach a movement level enabling them to live in society, including interactions with other people and safe living in the community. The interrater and intra-rater reliability were excellent (ICC, 0.948; 95% confidence interval [CI], 0.893 to 0.982 and ICC, 0.967; 95% CI, 0.933 to 0.989, respectively). Internal consistency was good (α=0.877). The convergent validity was moderate (r=0.646; p<0.001). Conclusions: The newly developed MPASS showed acceptable construct validity and high reliability. The MPASS is suitable for use in people with stroke, especially those who have been discharged and live in the community with the ability to initiate sitting.

• The Journal of Korean Medicine
• /
• v.17 no.1 s.31
• /
• pp.48-83
• /
• 1996

Our conclusions for function evaluation tools of Stroke patients are as follows. 1. Evaluating tools of Activities of Daily Living, Katz Index, Barthel Index, Modified Barthel Index have high validity and reliability because of ease of measuring, high accuracy, consistency, sensitivity and sufficient stastistics, but they mainly measure motor function except sense, mentation, language, and social conception. Therefore cerebrovascular disease and brain injury in trauma patients with lacked acknowledgement and sensation, we are not able to apply these tools. 2. PULSES Profile is a useful scale for measuring the patient's over-all status, upper and lower limb functions, sensory components, excretary functions, and intellectual and emotional adaptabilities. It is recognized as a good, useful tool to evaluate patient's whole function. 3. Motor Assessment Scale was designed to measure the progress of stroke patients. The scale was supplemented with upper arm function items. We believe that the Motor Assessment Scale could be a useful evaluation tool with inter-rater reliability ,test-retest reliability. 4. The existing evaluation tools, Katz Index, Barthel Index, Modified Barthel Index, PULSES Profile, Motor Assessment Scale, mainly measured the rehabilitational motor function of sequela of cerebrovascular patients. On the other hand CNS & INH stroke scale can measure cerebrovascular disease patient's neurologic deficits and over-all stautus, which are recognition ability, speech status, motor function, sensory function, activities of daily living. Those scales have been recognized as useful tools to measure function of cerebrovascular disease patients and have increased in use. 5. Every function evaluation tool was recognized to have some validity and inter-rater, test-retest reliability in items of each evaluation tool and total scores of each evaluation tools, but it is thought that none of these scales have been fully validated and proved reliable. Therefore afterward, the development of a highly reliable rating system may best be accomplished by a careful comparison of several tools, using the same patients and the same observers in order to choose the most reliable items from each. 6. Ideal evaluation tools must have the following conditions; (1) It should show the objective functional statues at the same time. (2) It should be repeated consecutively to know changed function status. (3) It should be easy to observe the treatment program. (4) It should have the same result with another rater to help rater exchange information with treatment team members. (5) It should be practical and simple. (6) The patient should not suffer from the observer.

• Journal of Korean Elementary Science Education
• /
• v.18 no.1
• /
• pp.65-73
• /
• 1999

The purpose of this study is tile development of tile observation Assessment criteria that is used to assess manipulative skills in elementary science. The procedures of developing observation Assessment criteria are as follows. First, we investigated the actual condition about science process skills assessment with tile questionnaire. Second, we selected 7 experimental apparatus through the analysis of the science textbooks. The selected experimental apparatus are dropper, alcohol lamp, thermometer, test tube, filtering device, able balance, graduated cylinder Third, the observation Assessment criteria arc developed through tile analysis of the related experimental textbooks, the demonstrations, and the questionnaire, Forth, tile validity is verified by science education specialists and graduate students who major in science education. Fifth, the first jilter-rater agreement is investigated in the result of the field with observation criteria of which the validity was verified. The second inter-rater agreement is investigated through the revision and the addition of the criteria with the low agreement. In result, the jilter-rater agreement ranged from 0.86 to 0.98. One of the major problems in observation assessment is the rater's subjective viewpoint. So, this research shows a more specified scale and criteria for the assessment. This suggests that the observation Assessment criteria developed in tile study satisfies the high reliability and validity requirements. Considering the results, this criteria call be used effectively for assessing manipulative skills of elementary school students.

• Physical Therapy Korea
• /
• v.10 no.1
• /
• pp.51-62
• /
• 2003

The purpose of this study was to compare the relative accuracy of a range of computer-based analysis with respect to EMG onset determined visually by an experienced examiner. Ten healthy students (6 male, 4 female) were recruited and three times randomly selected trials of isometric contraction of wrist flexion and extension were evaluated using four technique. These methods were compared which varied in terms of EMG processing, threshold value and the number of samples for which the mean must exceed the defined threshold, and beyond 7% of maximum amplitude. To identify determination of onset time, ICCs(Intraclass Correlation Coefficients) was used and inter-rater arid intra-rater reliability ranged good in visually derived onset values. The results of this study present that in wrist flexion and extension, the reliability of the inter and intra-examiner muscle contraction onset times through visual analysis showed beyond .971 with ICCs. The reliability of the muscle contraction onset time decision through visual reading, tested with computer analysis, showed a relationship of all the selected analysis methods with ICCs .859 and .871. The objective computer-based analysis comparing with visual reading at the same time is the effective and qualitative data analysis method, considering the specificity of each study method.