• Human Ecology Research
• /
• v.57 no.1
• /
• pp.109-125
• /
• 2019

Few studies have utilized observational methods in the field of couple research even though using self-report questionnaires is prone to the subjective biases of the reporter. This study validates the Marital Interaction Coding System-Global (MICS-G), a global version of the well-established microanalytic observational coding system, Marital Interaction Coding System (MICS). Participants in the study consisted of 30 married couples with varied levels of marital adjustment who visited one of the Healthy Family and Multicultural Family Support Centers in Seoul, either for couple therapy or the "Marriage Checkup"program. Ten-minute problem-solving discussions were rated by two undergraduate student raters who were trained for 10 hours. Interobserver agreement based on percentage agreement and intraclass correlation coefficients showed a high level of agreement between raters in establishing interrater reliability. Convergent validity was established by: correlations among marital adjustment, psychological aggression, mental health, and MICS-G categories of conflicts, validation, invalidation, facilitation, and withdrawal. MICS-G categories also were successful in discriminating between distressed and nondistressed couples, which provides evidence of discriminant validity for MICS-G. This study showed that MICS-G is a promising method for researchers to observe couple interactions in a more cost-effective way. Methodological issues and practical applications are also discussed.

• The korean journal of orthodontics
• /
• v.53 no.3
• /
• pp.163-174
• /
• 2023

Objective: This study evaluated the influence of various gingival displays on the esthetic perception in the presence of upper dental midline discrepancy. Methods: A smiling image of a male subject was altered digitally to produce five image series: normal smile (series A), decreased tooth show (series B), increased gingival show (series C), maxillary cant (series D), and asymmetric upper lip elevation (series E). In each image series, the midline was deviated to the right and left incrementally. A total of 210 raters (four professional groups and laypersons, n = 42 in each group) determined the midline deviation threshold and the attractiveness of midline position in each series. Results: The right and left thresholds were statistically similar for the symmetrical series (A, B, and C), while for series D, the right threshold was significantly lower. In most rater groups, the mean threshold order was: B > A > E > C > D. In all the series, the raters selected the coincident midline as the most attractive series except for series D, for which 1-2-mm deviations to the left were selected as the most attractive by almost all the groups. Conclusions: It is crucial to establish the coincident midline position in a symmetrical smile, especially when a gummy smile exists. In the asymmetrical gingival show, a coincident midline might not be the most esthetic midline position.

• KIPS Transactions on Software and Data Engineering
• /
• v.5 no.11
• /
• pp.527-534
• /
• 2016

The emergent information society requires the talent for creative thinking based on problem-solving skills and comprehensive thinking rather than simple memorization. Therefore, the Korean curriculum has also changed into the direction of the creative thinking through increasing short-answer questions that can determine the overall thinking of the students. However, their scoring results are a little bit inconsistency because scoring short-answer questions depends on the subjective scoring of human raters. In order to alleviate this point, an automated scoring system using a machine learning has been used as a scoring tool in overseas. Linguistically, Korean and English is totally different in the structure of the sentences. Thus, the automated scoring system used in English cannot be applied to Korean. In this paper, we introduce an automated scoring system for Korean short-answer questions using predictability and unanimity. We also verify the practicality of the automatic scoring system through the correlation coefficient between the results of the automated scoring system and those of human raters. In the experiment of this paper, the proposed system is evaluated for constructed-response items of Korean language, social studies, and science in the National Assessment of Educational Achievement. The analysis was used Pearson correlation coefficients and Kappa coefficient. Results of the experiment had showed a strong positive correlation with all the correlation coefficients at 0.7 or higher. Thus, the scoring results of the proposed scoring system are similar to those of human raters. Therefore, the automated scoring system should be found to be useful as a scoring tool.

• The Journal of Korean Medicine Ophthalmology and Otolaryngology and Dermatology
• /
• v.26 no.2
• /
• pp.10-18
• /
• 2013

Objectives : We performed a pilot study to investigate inter- and intra-rater reliability of pattern identification using nasal endoscopy for allergic rhinitis(AR). Methods : Eight experts of ophthalmology, otolaryngology and dermatology of Korean medicine evaluated 20 nasal endoscopy photograph cases of AR patients with pattern identification index using nasal endoscopy for AR including the nasal membrane color(pale / hyperemia), nasal membrane humidity(dryness / dampness), rhinorrhea(watery / yellow), and membrane edema (atrophic / edematous) on nasal endoscopy. Results : Intra-rater agreement(%) and Kappa coefficient was generally from 'moderate' to 'good'(% agreement: 73.13-90% / Kappa coefficient: 0.547-0.748). Inter-rater agreement(%) and Kappa coefficient was also from 'moderate' to 'good' (% agreement: 65-85% / Kappa: 0.475-0.778) except 'humidity(dryness / dampness)' item (% agreement: 55.98% / Kappa: 0.340). In findings of subgroup analysis according to affiliation of raters, Inter-rater agreement(%) and Kappa coefficient of raters in same affiliation was higher than inter-rater agreement(%) and Kappa coefficient of raters in different affiliation except 'dryness / dampness' item. Conclusions : It is necessary to improve objectivity and reproducibility of pattern identification using nasal endoscopy for allergic rhinitis(AR) through the development of detail-oriented criteria and enhanced training of clinicians with development of standard operating procedures(SOPs).

• Korean Journal of Culture and Social Issue
• /
• v.17 no.3
• /
• pp.261-285
• /
• 2011

In our studies, We defined the concept of social reputation and aimed to estimate the effect of social reputation on likability, trust, preference of interpersonal relationship. To accomplish these goal, photographs of 12 persons(six young unmarried man, six young unmarried women) volunteered were shown to 60 raters, and the one male and one female pictures of scores of facial attractiveness are nearest to median were selected as experimental figures in preliminary study. After this, We asked 260 raters to assess likability, trust, preference of interpersonal relationship after showing raters the chosen pictures of man and woman and telling scenario of positive, negative reputation about these two persons. The outcomes showed that in both men and women, likability, trust, preference of persons who gained positive reputation were significantly higher than persons gained negative reputation. Facial attractiveness was effect on likability, trust, and preference, but effect size of reputation was much higher on likability, interpersonal preference, especially trust level. Also, in three-way ANCOVA results, Woman has showed higher likability and trust than man under the condition of good reputation in both woman was rater and object to be assessed, but on the contrary, likability, trust, and preference of woman were lower than man in either woman was rater or object to be judged under the bad reputation,. Lastly, this study's implications and limitations were discussed.

• The Journal of the Korean dental association
• /
• v.54 no.11
• /
• pp.880-896
• /
• 2016

The comparison of the repeated measurements is often needed to see whether they agree sufficiently, when a measurement is repeated under identical conditions by different raters. Such investigations are often analyzed inappropriately, by using correlation coefficient. The purpose of this study is to introduce statistical methods for accessing the agreement of the repeated measurements, which include Bland-Altman plot, intra class correlation, Passing-Bablok regression and Cohen's kappa coefficient, and to show how to execute them using examples.

• Journal of the Korea Society of Computer and Information
• /
• v.20 no.12
• /
• pp.169-174
• /
• 2015

Permutation p-values are provided for the agreement measures for multivariate interval data among many raters. Three agreement measures, Berry and Mielke's measure, Janson and Olsson's measure, and Um's measure are described and compared. Exact and resampling permutation methods are utilized to compute p-values and empirical quantile limits for three measures. Comparisons of p-values demonstrate that resampling permutation methods provide close approximations to exact p-values, and Berry and Mielke's measure and Um's measure show similar performance in terms of measuring agreement.

• Journal of the Korean Data and Information Science Society
• /
• v.17 no.1
• /
• pp.77-86
• /
• 2006

This study is to investigate the error of prediction value with related variables from the recommended system and to examine the error of prediction value with related variables. To decrease the error on the collaborative recommended system on user based, this research explored the effects on the prediction related response pair between raters' demographic variables and Pearson's coefficient and sparsity. The result shows comparative analysis between existing error of prediction value and conditioned one.

• Physical Therapy Korea
• /
• v.24 no.3
• /
• pp.40-46
• /
• 2017

Background: The active knee extension (AKE) test commonly used to assess the flexibility of the hamstring muscles. Many researchers have tested the reliability of the AKE test; however, no published studies have examined the intrarater and interrater reliability of the AKE test using a PBU. Objects: The purpose of this study was to determine the intrarater and interrater reliability of the AKE test performed with a pressure biofeedback unit (PBU) on healthy subjects. Methods: Sixteen healthy male participants volunteered and gave informed consent to participate in this study. Two raters conducted AKE tests independently with a PBU. Each knee was measured twice, and the AKE testing was repeated one week after the first round of testing. Results: The interrater reliability's intraclass correlation coefficients ($ICC_{2,1}$) were .887~.986 for the right knees and .915~.988 for the left knees. In addition, the intrarater (test-retest) reliability ($ICC_{3,1}$) values ranged between .820~.915 and .820~.884 for Raters 1 and 2, respectively. The values for the standard error of mesurement were low for all tests ($.81{\sim}2.97^{\circ}$); the calculated minimum detectable change was $2.24{\sim}8.21^{\circ}$. Conclusion: These findings suggest that the AKE test performed with a PBU had excellent interrater and intrarater reliability for assessing hamstring flexibility in healthy young males.

• Phonetics and Speech Sciences
• /
• v.10 no.1
• /
• pp.75-81
• /
• 2018

Diplophonia is generally defined as the perception of more than one fundamental frequency component in a voice. Its perceptual aspect has traditionally been used to evaluate diplophonia because the perceptions can be easily evaluated, but there are limitations in the validity of the reliability of the intra- and inter-raters, examination situation, and variation of voice sample. Therefore, the purpose of this study is to confirm the reliability and accuracy of auditory perceptual evaluation by comparing non-invasive indirect assessment methods (sound waveform and EGG analysis), and to identify their usefulness with diplophonia. A total of 28 diplophonic voices and 39 non-periodic voices were assessed. Three raters assessed the diplophonia by performing an auditory perception evaluation and identifying the quasi-periodic perturbations of the acoustic waveform and EGG. Among the three discrimination methods, intra- and inter-rater reliability, sensitivity, specificity, accuracy, positive likelihood ratio, and negative likelihood ratio were examined, and the McNemar test was performed to compare the discriminant agreement. The accuracy of the auditory perceptual evaluation (86.57%) was not significantly different from that of sound waveform acoustic (88.06%), but it was significantly different from that of EGG (83.33%). The reading time (6.02 s) for the auditory perceptual evaluation was significantly different from that for sound waveform analysis (30.15 s) and EGG analysis (16.41 s). In the discrimination of diplophonia, auditory perceptual evaluation has sufficient reliability and accuracy as compared to sound waveform and EGG. Since immediate feedback is possible, auditory perceptual evaluation is more convenient. Therefore, it can continue to be used as a tool to discriminate diplophonia in clinical practice.