The Need for Level-based Criteria in the Assessment of Oral Proficiency

The purpose of this study is to identify the most important factor(s) that contribute(s) to the overall oral proficiency and to examine whether the factor(s) play(s) a different role depending on the proficiency level of learners. Learners were divided into novice group and advanced group and were asked to produce an oral recording of a story based on a comic strip. The recordings were transcribed and graded by three raters. According to the results, the raters attained high inter-rater reliability when assessing advanced learners. However, the reliability level became considerably lower in the assessment of novice learners. The best predictor of oral proficiency among novice learners was sociolinguistic competence and fluency, while grammatical competence and fluency were the strongest predictors for advanced learners. The results suggest the need for a separate assessment tool for different proficiency levels and the need for a different focus in the classroom depending on the learners' proficiency level.