• Journal of Oriental Neuropsychiatry
• /
• v.31 no.1
• /
• pp.1-12
• /
• 2020

Objectives: The purpose of this study was to evaluate the reliability and validity of the instrument on pattern identification for insomnia (PIT-Insomnia) and verify the correlation between PIT-Insomnia and psychological tests. Methods: Two evaluators examined the pattern identification of the participants who met insomnia disorder diagnostic criteria of the Diagnostic and Statistical Manual of Mental Disorder, Fifth Edition (DSM-5) and took the Insomnia Severity Index (ISI) score over 15 once manually and twice using the PIT-Insomnia to measure the inter-rater and test-retest reliability. We also conducted the following surveys: the Pittsburgh Sleep Quality Index (PSQI), the Korean version of Beck's depression inventory (K-BDI), the Korean version of the State-Trait Anxiety Inventory (STAI-K), the Korean Symptom checklist-95 (KSCL-95), and the EuroQol-5 dimension (EQ-5D), to measure concurrent validity and correlation between the PTI-Insomnia and psychological tests. Results: 1. The test-retest reliability analysis of the pattern identification results showed moderate agreement, and test-retest reliability analysis of each pattern identification score showed agreements from poor to moderate. 2. The inter-rater reliability analysis of the pattern identification results via manual showed slight agreement, when analysis was performed with calibration, the inter-rater reliability analysis of the pattern identification results via manual showed fair agreement. 3. The concordance analysis between results via manual and the PIT-Insomnia showed poor agreement, when the analysis was performed with calibration, concordance analysis showed fair agreement. 4. The concordance analysis between the PIT-Insomnia and the PSQI showed positive linear correlation. 5. The concordance analysis between the PIT-Insomnia and the PSQI, K-BDI, STAI-K, KSCL-95, and EQ-5D showed that non-interaction between the heart and kidney have positive linear correlation with the K-BDI, anxiety item of KSCL-95, dual deficiency of the heart-spleen have positive linear correlation with somatization item of KSCL-95, paranoia item of KSCL-95, heart deficiency with timidity have positive linear correlation with stress vulnerability item of KSCL-95, parania item of KSCL-95, phlegm-fire harassing the heart have positive linear correlation with K-BDI, paranoia item of KSCL-95, depressed liver qi transforming into fire have positive linear correlation with the anxiety item of KSCL-95, parania item of KSCL-95, all pattern identification have negative linear correlation with EQ-5D. Conclusions: The PIT-Insomnia has moderate agreement of reliability and reflects the severity of insomnia since it has some concurrent validity with the PSQI. There are some correlations between the PTI-Insomnia with specific psychological tests, so we could suggest it can be used appropriately in the clinical situation.

• Journal of Digital Contents Society
• /
• v.19 no.7
• /
• pp.1405-1412
• /
• 2018

Korean-PEQ (prosthesis evaluation questionnaire) was developed and verified its validity research but, evaluating its reliability applied to the lower extremity amputees have not been conducted. The aim of this study was to apply the Korean-PEQ for lower extremity amputees to evaluate the reliability. As a result, all of the items responding of Korean-PEQ shown to less than 15% ceiling effect and floor effect. therefore, the results were shown to be appropriate. Korean-PEQ reliability of each region of intra-class coefficient was shown to .719(95% CI .600~.811)~920(95%CI .890~.945) and the inter-region reliability was higher as 0.958. Item internal consistency Cronbach's ${\alpha}$ values are shown as higher .910.

• International Journal of Contents
• /
• v.15 no.4
• /
• pp.97-106
• /
• 2019

EMI (English as a Medium of Instruction) classes are now accepted policy at Korean universities, yet students often struggle with required academic English writings. The present study examined an EMI class that used direct instruction and access to online assistive English writing software. From preliminary analysis, 26 students expressed interest in how an EMI academic writing class could facilitate improved English writing skills. Study participants completed a survey on self-efficacy and learning needs and assignments for an EMI academic writing class. To establish inter-rater reliability, three trained raters assessed the written essays of students prior to and after instructional intervention. Fleiss' Kappas statistics showed moderate reliability. Students' opinions on the use of online software were also analysed. Paired t-test was run on the quality of students' pre- and post-instruction assignments, and there was significant difference in the rated scores. Self-efficacy was found to have moderate positive association with improved post-essay writing scores.

• Journal of the Korean Society of Physical Medicine
• /
• v.5 no.1
• /
• pp.89-99
• /
• 2010

Purpose : This study were to examine the test-retest and inter-rater reliability as well as the concurrent validity, and convergent validity of the 2 simplified Postural Assessment Scale for Stroke:5itmes-3Level(PASS:5i-3L) & Berg Balance Scale:7items

• English Language & Literature Teaching
• /
• v.15 no.2
• /
• pp.169-184
• /
• 2009

The purpose of this study is to identify the most important factor(s) that contribute(s) to the overall oral proficiency and to examine whether the factor(s) play(s) a different role depending on the proficiency level of learners. Learners were divided into novice group and advanced group and were asked to produce an oral recording of a story based on a comic strip. The recordings were transcribed and graded by three raters. According to the results, the raters attained high inter-rater reliability when assessing advanced learners. However, the reliability level became considerably lower in the assessment of novice learners. The best predictor of oral proficiency among novice learners was sociolinguistic competence and fluency, while grammatical competence and fluency were the strongest predictors for advanced learners. The results suggest the need for a separate assessment tool for different proficiency levels and the need for a different focus in the classroom depending on the learners' proficiency level.

• The Journal of the Korea Contents Association
• /
• v.11 no.9
• /
• pp.407-413
• /
• 2011

Distinguishing aphasia in stroke patients and observing the subtle linguistic characteristics associated with it primarily requires the use of instruments that provide reliable assessment results. Additionally, examiners should be fully aware of how to use those instruments. This study examined 46 stroke patients for aphasia and assessed the reliability of their diagnoses according to examiners whose medical fields were different from each other. Furthermore, a comparison was made between the reliability before training and that after training. To this end, 46 stroke patients were tested for aphasia and in terms of their speech disorder degree by 3 groups, each of which consisted of 12 professionals (3 SLP, 3 neurologist, and 3 nurse). In the result, a rating of 'acceptable' was given for speech intelligibility tasks and the voice quality of /ah-/ prolongation, and other sub-tests were marked as 'good-excellent' by the experts with different areas of medical expertise. For the tasks marked as 'acceptable', the subjects were video-trained for 3 weeks and the differences were compared before and after their training. Consequently, the differences in the examiners' ratings in the speech intelligibility tasks showed a significant decrease and the accuracy of their voice quality ratings showed a significant increase. In the result of research on the correlation between the accuracy of the sub-test ratings and the amount of clinic experience, speech therapists developed more accuracy in rating a picture description task and a speech intelligibility task as their experience accumulated. Meanwhile, doctors and nurses showed more accurate ratings in picture description tasks with greater clinical experience. The results of this study suggest that assessing the neurologic-communicative disorders of stroke patients requires ongoing training and experience, especially for speech disorders. It was also found that the rating reliability in this case could be improved by training.

• Journal of rehabilitation welfare engineering & assistive technology
• /
• v.11 no.2
• /
• pp.157-163
• /
• 2017

The aim of the study is to examine the reliability and validity of UFOV, which is a visual driving evaluation tool that has been proven to be reliable and valid in western countries, for the purpose of adapting the tool in a systematic manner to the South Korean population. Two evaluator assessed 23 healthy and 19 stroke patients with UFOV, Trail Making Test A & B(TMT A & B) and Motor Free Visual Perception Test(MVPT) from 7 October 2014 to 25 November, 2014. The researcher analyzed inter-rater reliability, correlation between raters of UFOV with Intraclass correlation coefficient, test-retest reliablility, UFOV with spearman correlation coefficient, concurrent validity, UFOV, TMT A & B and MVPT with spearman correlation coefficient, and discriminative validity, comparison mean scores of UFOV between groups, healthy and stroke with Mann-Whitney U test. UFOV score of participants with stroke had lower compared to the healthy control group. The inter-rater reliability(p<.001), test-retest reliability(p<.01) and concurrent validity(p<.01) was statistically significant. Also discriminant validity was statistically significant(p<.001). Based on this study, Use of UFOV for drivers at risk is essential to prevent future traffic accidents and support driving rehabilitation.

• Journal of the Korean Society of Laryngology, Phoniatrics and Logopedics
• /
• v.18 no.2
• /
• pp.96-101
• /
• 2007

Laryngopharyngeal reflux (LPR) is the retrograde movement of gastric contents into the larynx, pharynx, and upper aero-digestive tract. LPR differs from gastroesophageal reflux in that it is often not associated with heartburn and regurgitation symptoms. Otolaryngological manifestations of acid reflux include a wide range of pharyngeal and laryngeal symptoms. Belafsky et al. developed a useful self-administered tool, the reflux symptom index (RSI), for assessing the degree of LPR symptoms. Patients are asked to use a 0 to 5 point scale to grade the following symptoms: 1) hoarseness or voice problems; 2) throat clearing; 3) excess throat mucus or postnasal drip ; 4) difficulty swallowing; 5) coughing after eating or lying down; 6) breathing difficulties ; 7) troublesome or annoying cough; 8) sensation of something sticking or a lump in the throat; 9) heartburn, chest pain, indigestion or stomach acid coming up. A RSI score greater than 13 is considered abnormal. As there is no validated instrument to document the physical findings and severity of LPR, Belafsky et al. developed an eight-item clinical severity scale for judging laryngoscopic finding, the reflux finding score (RFS). They rated eight LPR-associated findings on a scale from 0 to 4 : subglottic edema, ventricular obliteration, erythema/hyperemia, vocal-fold edema, diffuse laryngeal edema, posterior commissure hypertrophy, granuloma/granulation tissue, and thick endolaryngeal mucus. A RFS score of greater than 7 was found to suggest LPR-associated laryngitis. Although both indices (RSI and RFS) are widely used, there is some controversy about their validity (sensitivity and specificity) and reliability (intra-rater and inter-rater) in LPR diagnosis and treatment. We discuss the validity and reliability of RSI and RFS with literature review.

• Journal of the Korean Data and Information Science Society
• /
• v.23 no.1
• /
• pp.25-37
• /
• 2012

Many statistics, such as Cohen's (1960) ${\kappa}$, Scott's (1955) ${\pi}$, and Park and Park's (2007) H have been proposed as measures of agreement to represent inter-rater reliability. This study compared bias, SE, MSE, and CV of the measures of agreement with nominal and ordinal categories in the balanced marginal distributions, and those with nominal categories in the two paradoxical situations. As a result, in all cases, AC1and Hhad smaller SE and CV.

• Journal of Korean Dental Hygiene Science
• /
• v.4 no.1
• /
• pp.53-64
• /
• 2021

Background: The elderly in long-term care facilities tend to have a diminished oral health status, with a high prevalence of dental caries and periodontal disease, as reduced cognitive function, joint mobility, and gait ability hinder the performance of oral hygiene. To improve the oral health of the elderly at long-term care facilities, it is necessary to have precise guidelines for oral health care and assessment; however, such guidelines are not readily available. Therefore, the present study aimed to develop an oral care assessment tool with verified reliability and validity. Methods: The participants in this study were 100 elderly patients at a care facilities and 10 clinical dental hygienists. Collected data were analyzed using the descriptive statistics, content validity index, and inter-rater reliability, as well as the analyses of intra-class correlation coefficient. Results: After a review of relevant literature, a preliminary questionnaire comprised of seven questions related to the evaluation of oral health was formed. After revising and supplementing the questions through a content validity test, a total of nine questions were selected. Conclusion: The novel assessment tool developed for the present study is anticipated to allow analyses of the level of problems related to oral health care before routine and professional care. Moreover, regular oral health status check-ups will enable the early diagnosis and treatment of diseases.