• Journal of The Korean Association For Science Education
• /
• v.30 no.2
• /
• pp.275-290
• /
• 2010

In this study, reliabilities of holistic scoring method and analytic scoring method were analyzed in performance assessments of middle school students' science investigation activity. Reliabilities of 2, 3, and 4~7-level rubric ratings for analytic scoring methods were compared to figure out optimized numbers of rubric ratings. Two trained raters rated four activity sheets of 60 students by two rating methods and three kinds of rubric ratings. Internal consistency reliabilities of holistic scoring methods were higher than those of analytic scoring methods, while intrarater reliabilities of analytic scoring were higher than those of holistic scoring methods. Internal consistency reliabilities and intra-rater reliabilities of 3-level rubric rating showed similar patterns of 4~7-level rubric ratings. But students' discriminations, item difficulties and item-response curves showed that the 3-level rubric ratings was reliable. These results suggest that holistic scoring method could be adapted to increase internal consistency reliabilities with improvement in intra-rater reliabilities by rater's conferences. Also, the 3-level rubric rating would be enough for good reliability in case of adapting analytic scoring methods.

• Journal of The Korean Association For Science Education
• /
• v.19 no.4
• /
• pp.507-515
• /
• 1999

This study is to develope the performance scoring rubrics for the inquiry-based experiments of general chemistry course in the college of education. Two types of analytic scoring rubrics have been developed for nine different experiments. The first one is to assess scientific process skills from the written experimental reports. These analytic scoring rubrics include seven process skills selected from the Lawson's 'creative and critical thinking skills' and other known process skills. The second one is to assess the individual manipulative skills and experimental attitudes through direct observations by the teacher. The content validity of all scoring rubrics was testified by six science educators. Also the inter-scorer reliability of analytic scoring rubrics administered on the students' experimental reports was examined. The correlation coefficient between the scores obtained from the experiments and those of the written test for theoretical knowledges was found to be r=.663(p <.01). From the variance($r^2$=.440), we would say indirectly that the 56% of this experimental assessment does not overlap with the theoretical knowledges test and assesses students' science process skills, manipulative skills, and attitudes.

• Journal of Korean Society of Rural Planning
• /
• v.16 no.3
• /
• pp.67-76
• /
• 2010

The objective of this study is to investigate a priority and selection of agricultural specialization items from Daegu and Gyeongbuk region by using analytical tools such as, the Analytic Hierarchy Process(AHP) and the Scoring Method. The results of this study are as follows; 1) Five major items were selected for each region based on Daegu and Gyeongbuk regional agricultural performances. 2) The results of AHP analysis, Daegu and Gyeongbuk's agricultural experts evaluated the weight of three decision-elements(objectiveness 0.35, efficiency 0.341, identity 0.309) and seven decision-elements. 3) Results of AHP and Scoring Method analysis, Investigated a priority of agricultural specialization 3 items in 25 cities and districts Daegu and Gyeongbuk regional agricultural performances. 4) Finally, The results of this study will provide basic information on agricultural specialization policy in Daegu and Gyeongbuk region.

• Journal of Korean Society of Rural Planning
• /
• v.12 no.4 s.33
• /
• pp.1-11
• /
• 2006

Recently many administrative institutes try to improve the viability of rural villages. For increasing the viability, not only infrastructures but internal vitality is necessary in rural villages. Nonetheless, most of governmental projects have been focused on infrastructures. For this reason, RDA(Rural Development Administration) designed and performed the RHL(Rural Healthy and Longevity village) project. This RHL project is not easy to evaluate the outcome because it consists of very intangible project items. In this paper, we developed a scoring model to evaluate the result of the RHL project. The scoring model based on DEA(Data Envelopment Analysis) was suggested to evaluate the quantity of personal activities in each village. Personal activities are classified into five categories: regional life, social life, productive life, outdoor life and indoor life. Evaluating indices of each category are developed and weighting values are evaluated by AHP(Analytic Hierarchy Process). The developed model was applied to Kumsan village and examined its applicability.

• Journal of Educational Research in Mathematics
• /
• v.8 no.1
• /
• pp.217-235
• /
• 1998

The purpose of this study is to define the altenative assessment and to suggest the method of scoring system. Alternative assessment includes any type of assessment in which student create reponses to a question rather than choosing a responses form given list( as for multiple choice, true/false, or matching). Alternative assessment can includes short answer questions, essay, performances, oral presentation, demonstrations, exhibitions, portfolios, and etc. To evaluate the each type of assessment, we can apply the method of holistic scoring and analytic scoring system. Also we have to concern the type of scoring mechanism directly relate to what we want to assess, our purpose for assessment fitting into the educational enterprise. Before applying the alternative assessment in our classroom, we need to step back and reconsider all our design features and teachers' responsibility.

• Proceedings of the KOSOMBE Conference
• /
• v.1997 no.05
• /
• pp.199-202
• /
• 1997

We are developing computer integrated polysomnography system. This system integrates conventional polysomnography with computer for data management, automatic analysis, scoring, and data transmission. In the first stage, we have developed the signal interface and user interface for the manual scoring and data management. For the automatic scoring of sleep stage, we have developed the protocol and have applied the analytic method in its primitive form. In the second stage we will develope a partially automatic scoring system, and finalize the fully automatic system in the final third stage.

• English Language & Literature Teaching
• /
• v.12 no.1
• /
• pp.221-238
• /
• 2006

This study investigates the reliability of plagiarism measurement in the ESL essay writing context. The current study aims to address the answers to the following research questions: 1) How does plagiarism measurement affect test reliability in a psychometric view? and 2) how do raters conceive the plagiarism in their analytic scoring? This study uses the mixed-methodology that crosses quantitative-qualitative techniques. Thirty eight international students took an ESL placement writing test offered by the University of Illinois. Two native expert raters rated students' essays in terms of 5 analytic features (organization, content, language use, source use, plagiarism) and made a holistic score using a scoring benchmark. For research question 1, the current study, using G-theory and Multi-facet Rasch model, found that plagiarism measurement threatened test reliability. For research question 2, two native raters and one non-native rater in their email correspondences responded that plagiarism was not a valid analytic area to be measured in a large-scale writing test. They viewed the plagiarism as a difficult measurement are. In conclusion, this study proposes that a systematic training program for avoiding plagiarism should be given to students. In addition, this study suggested that plagiarism is measured reliably in the small-scale classroom test.

• Journal of Society of Preventive Korean Medicine
• /
• v.13 no.1
• /
• pp.105-115
• /
• 2009

Background: The resource-based relative value scale has been applied to Korean medicine services as part of the National Health Insurance, yet the proper scoring method for physician's work needs to be developed. Objectives: To measure physician's work for Korean medicine services using Analytical Hierarchy Process(AHP) and to compare results abstracted by AHP with those by Magnitude Estimation method. Methods: We developed the AHP questionnaires for Korean Medicine services and administered it to 12 Korean medicine insurance experts. Weighted scores were calculated and were converted into physician's work scores. Results: In AHP measurement, the physician's work scores for acupuncture activities, especially special acupunctures, far increased, while those for diagnosis and basic consultations decreased. Variations in the scores enlarged. Conclusions: As an alternative, AHP method can be considered for measuring the physician's work for Korean medicine services with careful management of scoring variation.

• Communications of Mathematical Education
• /
• v.26 no.3
• /
• pp.251-271
• /
• 2012

The purpose of this study is: 1) to determine error sources and the effects of each error source, 2) to investigate optimal measuring conditions from holistic and analytic scoring methods, and 3) to compare the value of reliability between Cronbach's alpha and the generalizability coefficient in self-introduction letter and teacher's recommendation letter based on the generalizability theory in identification of mathematical gifted students by observations and nominations. Data of this study were collected from the science education institute for the gifted attached to the university located within in a capital city for the 2011 academic year. Scores form two raters using holistic and analytic scoring methods in both assessment types were used. The results of this study were as follows. First, as to both assessment types, error sources for people were relatively large regardless of scoring methods. However, error sources for raters in holistic scoring methods had a more significant impact than those of analytic scoring methods. Second, to set optimal measuring conditions in the self-introduction letter and teacher's recommendation letter, if we fixed the number of raters into 2 based on holistic scoring methods, at least 5 and 10 content domains were needed, respectively. In addition, the number of items in teacher's recommendation letter should be more than 3 when we fixed the number of content domains into 4, and the number of items in self-introduction letter should be more than 8 when we fixed the number of content domains into 6 using analytic scoring methods. Third, Cronbach's alpha having only a single source of errors was higher than the generalizability coefficient regardless of assessment types and scoring methods. Hence we recommend that generalizability coefficient based on various error sources such as raters, content domains, and items should be considered to keep a satisfactory level of reliability in both assessment types.

• Journal of the Korea Safety Management & Science
• /
• v.10 no.3
• /
• pp.193-201
• /
• 2008

As The development and commercialization of advanced technologies will depend increasingly on efficient technology transfer and technology trading systems and the knowledge economics grows rapidly, the value of intangible assets is more emphasized in business nowadays. Intangible assets include intellectual capital and intellectual property. This requires the development of technology markets or exchanges and hence a reliable technology valuation methodology. Our focus is on the business aspects of technology' contribution and we propose an objective scoring system, which is implemented by ANP, for Technology valuation of enterprise. As a result, This study intends to evaluate the contribution of an individual technology in intangible assets by the Analytic Network Process(ANP).