• The Korean Journal of Applied Statistics
• /
• v.9 no.2
• /
• pp.45-53
• /
• 1996

Essay-writing, first introduced to Korean university entrance exams in 1994, is gaining its weight year by year. Especially from 1997 when Nationwide Education Reform System begins, it will be a key component of student selection criteria at Korean universities. Essay-writing's future, however, will not be that smooth unless it shows necessary validity and reliability. This study is on reliability of Essay-writing scoring, mainly from the experience of University K case. To secure solid reliability in Essay-writing scoring for the 1995 University Entrance Exam, the authors started research from the 1994 Autumn Pre-exam which was administred to potential applicants of University K following year. Total of 1,254 students took Essay-writing exam and, subsequently, their essays were graded by two professors independently. The result was not so good. The correlation between two scores was 0.27[0.54] with Cronbach alpha 0.43[0.70] for Humanity-Social Science [Natural Science-Engineering] field. So, some action for reliability improvement was inevitable. The authors considered and investigated following two alternatives. Alternative 1 [A1] : Essays are to be graded three tiems independently at the 1995 University K Entrance Exam. Scores will be given as the average of three scores. Alternative 2 [A2] : Essays are to be graded twice independently, followed by a possible third grading only if two gradings show "significant" defference. Scores are given as the third score if done or the average of first two scores otherwise.otherwise.

• Korean Journal of Cognitive Science
• /
• v.27 no.2
• /
• pp.221-245
• /
• 2016

Peer assessment refers to having students, rather than the instructor, make assessments of one another's work. Peer assessment is often used as a tool to train writing skills or a tool to apply or extend learning in higher education. Park(2016) recently proposed a system which utilizes peer assessment as a part of preparatory activity for college courses. Before weekly class, students studied given material on their own, wrote a one page essay on a given question based on their reading, and assessed the essays of other students. In this study, the system was implemented in undergraduate courses at S University over 2 semesters and the results were analyzed. The reliability of weekly scores given by students was not very high, but the correlation was high between the cumulative scores given by students across weeks and the scores of the end of the term paper assessed by the instructor. Based on these findings, the possibility of utilizing the results of the peer assessments as part of the final grades was discussed.

• Journal of Elementary Mathematics Education in Korea
• /
• v.13 no.2
• /
• pp.231-245
• /
• 2009

Almost every children are naturally non competitive, so they have test anxiety. Because grading in paper-pencil tests is related to competition with their friends. Intense competition should not be a part of the classroom environment. This study intends to reflect current standardized tests(summative evaluations) in an elementary school mathematics(2008) and reconstructed it's problem. Teachers should provide students with rich situational problem-solving opportunities. Therefore we conclude "summative evaluation are consisted of some large problem type which contain some subproblems, or from easy(simple) problem with relation to difficult problem)".

• Communications of Mathematical Education
• /
• v.26 no.3
• /
• pp.251-271
• /
• 2012

The purpose of this study is: 1) to determine error sources and the effects of each error source, 2) to investigate optimal measuring conditions from holistic and analytic scoring methods, and 3) to compare the value of reliability between Cronbach's alpha and the generalizability coefficient in self-introduction letter and teacher's recommendation letter based on the generalizability theory in identification of mathematical gifted students by observations and nominations. Data of this study were collected from the science education institute for the gifted attached to the university located within in a capital city for the 2011 academic year. Scores form two raters using holistic and analytic scoring methods in both assessment types were used. The results of this study were as follows. First, as to both assessment types, error sources for people were relatively large regardless of scoring methods. However, error sources for raters in holistic scoring methods had a more significant impact than those of analytic scoring methods. Second, to set optimal measuring conditions in the self-introduction letter and teacher's recommendation letter, if we fixed the number of raters into 2 based on holistic scoring methods, at least 5 and 10 content domains were needed, respectively. In addition, the number of items in teacher's recommendation letter should be more than 3 when we fixed the number of content domains into 4, and the number of items in self-introduction letter should be more than 8 when we fixed the number of content domains into 6 using analytic scoring methods. Third, Cronbach's alpha having only a single source of errors was higher than the generalizability coefficient regardless of assessment types and scoring methods. Hence we recommend that generalizability coefficient based on various error sources such as raters, content domains, and items should be considered to keep a satisfactory level of reliability in both assessment types.

• Science of Emotion and Sensibility
• /
• v.11 no.4
• /
• pp.553-564
• /
• 2008

Generalizability theory was employed to examine how the reliability of polygraph test is affected by the number of relevant questions, the number of repeated tests (the number of of charts), and the number of raters(scorers). The data consisted of the results of the polygraph tests administered to 31 crime suspects. The sample was drawn from the real polygraph tests based on Backster ZCT and archived by the Prosecutor's Office of the Republic of Korea. The numerical scores assigned by thirteen raters to the test charts were analyzed to determine the generalizability of the scores. The largest variance component was accounted for by the examinee factor(43.97%) and the residual variance component was 16.84% of the total variance. The variance component due to the interaction between the examinee and the chart factors was 12.17% and the variance component due to the three way interaction of the examinee, the repeated test, and the relevant question factors was 10.31%. The generalizability coefficient for the current measurement procedure as practiced by the Korean Prosecutor's Office was 0.74 which suggests that the current procedure is acceptable. However, measurement procedures with the combination of more than two relevant questions, more than three repeated tests, and more than two raters were generally found to yield generalizability coefficients larger than 0.80. Therefore, such procedures need to be considered seriously in order to significantly improve the reliability of polygraph test.

• Journal of the Korean Data and Information Science Society
• /
• v.27 no.2
• /
• pp.395-407
• /
• 2016

This study aims to apply the G-theory for estimation of reliability of evaluation scores between raters on Taekwondo Poomsae rating categories. Selecting a number of game days and raters as multiple error sources, we analyzed the error sources caused by relative magnitude of error variances of interaction between the factors and proceeded with D-study based on the results of G-study for optimal determination of measurement condition. The results showed below. The estimated outcomes of variance component for accuracy among the Taekwondo Poomsae categories with G-theory showed that impact of error was the biggest influence factor in raters conditions and in order of interaction in subjects and between subjects, also impact of variance component estimation error on expression category was the major influence factor in interaction and in order of the between subjects and raters. Finally, the result of generalizability coefficient estimation via D-study showed that measurement condition of optimal level depend on the number of raters was 8 persons of raters on accuracy category, and stable reliability on expression category was gained when the raters were 7 persons.

• Journal of Korean Home Economics Education Association
• /
• v.32 no.3
• /
• pp.135-159
• /
• 2020

The purpose of this study was to develop items of an essay response assessment that could align with the instructions and assessments in the high school home economics curriculum. The contents of the study were as follows. First, to establish an assessment plan, 14 achievement standards were analyzed in the assessment area, and the elements of the questions were developed including the content elements of a total of 29 questions. Second, to develop the assessment tools, preliminary questions suited to the structure of essay questions were developed, and the method of presenting data and scoring criteria to be utilized in the questions was selected. Third, to prepare the answers and the scoring criteria tables, the answers to the sample questions for each score were prepared in form of a scoring criteria table, and the objectives of the assessment, the scoring items, and the scores for each item were reviewed. Fourth, the developed questions and answers were revised and supplemented by teachers of the professional learning community through preliminary and mutual review on the components of the questions, the embodiment of the assessment objectives, the implementation of the assessment intent, and the grading. This study can be used as a foundational study for the development of essay-type questions and scoring criteria in essay assessment in the field of education. Furthermore, the results of this study could help teachers enhance their learners' ability to apply knowledge in the future.

• Journal of the Korean Academy of Child and Adolescent Psychiatry
• /
• v.5 no.1
• /
• pp.54-69
• /
• 1994

The purpose of present study was to develop the Korean Version of Luria-Nebraska Neuropsychological Battery for Children(LNNB-C), to examine the reliability of it, and to establish the norms for determining the probability of brain damage. The normative group used to standardize the Korean version of LNNB-C was composed of 147 children between the age of 8 and 12(body 74, girl 73). The clinical group consisted of 19 brain damaged, 16 ADHD, and 16 psychiatric controls. The inter-scorer reliability was 96.3%, indicating that the stability of the scoring system for the Korean version of LNNB-C is good. The reliability coefficients(Cronbach's ${\alpha}$) of LNNB-C scales were ranged .51 to .91, which are similar to those of original LNNB-C. To establish the norms for detecting brain damage, the means and standard deviations for normative group were used to calculate T-scores for each scale. To determine a critical level that could successfully predict a normal child's average score at a given age, first the average score of normative group was calculated, and this score was then entered a regression equation with age to predict the average(baseline) acore. Finally, some issues on constructing the Korean version of LNNB-C and the cultural differences between Korean and American children in performing LNNB-C were discussed.

• Journal of the Korean Society of Marine Environment & Safety
• /
• v.25 no.4
• /
• pp.504-510
• /
• 2019

Practical test items were analyzed to extend the existing scoring method for boat operator licenses to an electronic scoring method. We have attempted to digitize the method within the current practical test system scope and have developed an electronic scoring system using LiDAR sensors and WAVE communication. The results of the study are as follows; the first, the scoring data entered into the LiDAR and examiner score device on the boat were transferred from an integrated processing unit to a land control center through WAVE communication. The system was constructed and verified to store and manage examinee data. Second, when testing the meandering task, accurate distance measurement was achieved by using LiDAR instead of visually observing the stick (3 m), and an accurate distance was displayed through the examiner score device quickly. Finally, we confirmed that it is possible to smoothly transmit and process the WAVE communication used to transfer the score data acquired from the boat to the monitoring center at a high speed without loss.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2015.01a
• /
• pp.337-338
• /
• 2015

최근 다양한 모바일 기기를 통한 여러 가지 문제지가 등장하고 있다. 본 논문에서는 여러 종류의 기출문제를 풀고 답을 확인할 수 있는 문제지를 개발하고자 한다. 먼저 로그인 폼을 만들어 회원정보를 담고 여러 가지 문제를 입력하여 문제지를 만들고 문제를 다 풀면 자동으로 채점을 하여 일일이 점수를 계산하는 시간낭비를 줄였다.